2023-02-13 00:54:32,792 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 00:54:33,039 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5 2023-02-13 00:54:33,053 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 00:54:33,054 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c, deleteOnExit=true 2023-02-13 00:54:33,054 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 00:54:33,054 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/test.cache.data in system properties and HBase conf 2023-02-13 00:54:33,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 00:54:33,055 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/hadoop.log.dir in system properties and HBase conf 2023-02-13 00:54:33,056 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 00:54:33,056 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 00:54:33,056 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 00:54:33,171 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 00:54:33,549 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 00:54:33,553 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 00:54:33,553 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 00:54:33,553 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 00:54:33,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 00:54:33,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 00:54:33,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 00:54:33,554 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 00:54:33,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 00:54:33,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 00:54:33,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/nfs.dump.dir in system properties and HBase conf 2023-02-13 00:54:33,555 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/java.io.tmpdir in system properties and HBase conf 2023-02-13 00:54:33,556 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 00:54:33,556 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 00:54:33,556 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 00:54:34,041 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 00:54:34,046 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 00:54:34,317 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 00:54:34,466 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 00:54:34,480 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:54:34,515 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:54:34,546 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/java.io.tmpdir/Jetty_localhost_38911_hdfs____.l11cjm/webapp 2023-02-13 00:54:34,668 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38911 2023-02-13 00:54:34,684 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 00:54:34,685 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 00:54:35,166 WARN [Listener at localhost/41999] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:54:35,233 WARN [Listener at localhost/41999] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 00:54:35,256 WARN [Listener at localhost/41999] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:54:35,263 INFO [Listener at localhost/41999] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:54:35,267 INFO [Listener at localhost/41999] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/java.io.tmpdir/Jetty_localhost_33851_datanode____.nu0bl1/webapp 2023-02-13 00:54:35,362 INFO [Listener at localhost/41999] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33851 2023-02-13 00:54:35,662 WARN [Listener at localhost/45053] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:54:35,672 WARN [Listener at localhost/45053] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 00:54:35,675 WARN [Listener at localhost/45053] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:54:35,677 INFO [Listener at localhost/45053] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:54:35,684 INFO [Listener at localhost/45053] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/java.io.tmpdir/Jetty_localhost_33933_datanode____8467hc/webapp 2023-02-13 00:54:35,779 INFO [Listener at localhost/45053] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33933 2023-02-13 00:54:35,789 WARN [Listener at localhost/44869] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:54:35,809 WARN [Listener at localhost/44869] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 00:54:35,812 WARN [Listener at localhost/44869] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 00:54:35,813 INFO [Listener at localhost/44869] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 00:54:35,821 INFO [Listener at localhost/44869] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/java.io.tmpdir/Jetty_localhost_34223_datanode____.ofgso7/webapp 2023-02-13 00:54:35,926 INFO [Listener at localhost/44869] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34223 2023-02-13 00:54:35,933 WARN [Listener at localhost/43763] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 00:54:36,118 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd0cd26d830b4f9c9: Processing first storage report for DS-56a297bc-d869-4093-96c2-e30c2af4c59b from datanode 4e877f44-49cc-41c6-9a12-77167167aa83 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd0cd26d830b4f9c9: from storage DS-56a297bc-d869-4093-96c2-e30c2af4c59b node DatanodeRegistration(127.0.0.1:38789, datanodeUuid=4e877f44-49cc-41c6-9a12-77167167aa83, infoPort=37021, infoSecurePort=0, ipcPort=44869, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd8426d515c26aa7d: Processing first storage report for DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5 from datanode 5c1a259a-c33c-46e5-8622-2360d3fa2b07 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd8426d515c26aa7d: from storage DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5 node DatanodeRegistration(127.0.0.1:36179, datanodeUuid=5c1a259a-c33c-46e5-8622-2360d3fa2b07, infoPort=44745, infoSecurePort=0, ipcPort=45053, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb10f8749d42249dd: Processing first storage report for DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69 from datanode f25dc2b5-f59c-472e-873a-01e1c6fd94f9 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb10f8749d42249dd: from storage DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69 node DatanodeRegistration(127.0.0.1:44837, datanodeUuid=f25dc2b5-f59c-472e-873a-01e1c6fd94f9, infoPort=37531, infoSecurePort=0, ipcPort=43763, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd0cd26d830b4f9c9: Processing first storage report for DS-0d361510-0eb6-4b6a-bc22-8e1282951016 from datanode 4e877f44-49cc-41c6-9a12-77167167aa83 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd0cd26d830b4f9c9: from storage DS-0d361510-0eb6-4b6a-bc22-8e1282951016 node DatanodeRegistration(127.0.0.1:38789, datanodeUuid=4e877f44-49cc-41c6-9a12-77167167aa83, infoPort=37021, infoSecurePort=0, ipcPort=44869, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,120 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd8426d515c26aa7d: Processing first storage report for DS-585d99ff-3c7a-457e-a348-701ec9d22ea2 from datanode 5c1a259a-c33c-46e5-8622-2360d3fa2b07 2023-02-13 00:54:36,121 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd8426d515c26aa7d: from storage DS-585d99ff-3c7a-457e-a348-701ec9d22ea2 node DatanodeRegistration(127.0.0.1:36179, datanodeUuid=5c1a259a-c33c-46e5-8622-2360d3fa2b07, infoPort=44745, infoSecurePort=0, ipcPort=45053, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,121 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb10f8749d42249dd: Processing first storage report for DS-5be03c60-6de1-4ef3-a320-a0d811bf4e46 from datanode f25dc2b5-f59c-472e-873a-01e1c6fd94f9 2023-02-13 00:54:36,121 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb10f8749d42249dd: from storage DS-5be03c60-6de1-4ef3-a320-a0d811bf4e46 node DatanodeRegistration(127.0.0.1:44837, datanodeUuid=f25dc2b5-f59c-472e-873a-01e1c6fd94f9, infoPort=37531, infoSecurePort=0, ipcPort=43763, storageInfo=lv=-57;cid=testClusterID;nsid=1904460520;c=1676249674116), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 00:54:36,308 DEBUG [Listener at localhost/43763] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5 2023-02-13 00:54:36,374 INFO [Listener at localhost/43763] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/zookeeper_0, clientPort=63852, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 00:54:36,391 INFO [Listener at localhost/43763] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=63852 2023-02-13 00:54:36,402 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:36,405 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:37,082 INFO [Listener at localhost/43763] util.FSUtils(479): Created version file at hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac with version=8 2023-02-13 00:54:37,083 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/hbase-staging 2023-02-13 00:54:37,410 INFO [Listener at localhost/43763] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 00:54:37,839 INFO [Listener at localhost/43763] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 00:54:37,871 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:37,871 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:37,872 INFO [Listener at localhost/43763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:54:37,872 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:37,872 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:54:38,025 INFO [Listener at localhost/43763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:54:38,095 DEBUG [Listener at localhost/43763] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 00:54:38,187 INFO [Listener at localhost/43763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43619 2023-02-13 00:54:38,197 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,198 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,218 INFO [Listener at localhost/43763] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43619 connecting to ZooKeeper ensemble=127.0.0.1:63852 2023-02-13 00:54:38,260 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:436190x0, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:54:38,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43619-0x1004b2c240b0000 connected 2023-02-13 00:54:38,286 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:54:38,287 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:54:38,290 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:54:38,298 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43619 2023-02-13 00:54:38,298 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43619 2023-02-13 00:54:38,299 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43619 2023-02-13 00:54:38,299 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43619 2023-02-13 00:54:38,299 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43619 2023-02-13 00:54:38,305 INFO [Listener at localhost/43763] master.HMaster(439): hbase.rootdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac, hbase.cluster.distributed=false 2023-02-13 00:54:38,373 INFO [Listener at localhost/43763] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 00:54:38,373 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,373 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,373 INFO [Listener at localhost/43763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:54:38,373 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,374 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:54:38,378 INFO [Listener at localhost/43763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:54:38,381 INFO [Listener at localhost/43763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43727 2023-02-13 00:54:38,383 INFO [Listener at localhost/43763] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 00:54:38,389 DEBUG [Listener at localhost/43763] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 00:54:38,391 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,392 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,394 INFO [Listener at localhost/43763] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43727 connecting to ZooKeeper ensemble=127.0.0.1:63852 2023-02-13 00:54:38,398 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:437270x0, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:54:38,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43727-0x1004b2c240b0001 connected 2023-02-13 00:54:38,399 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:54:38,400 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:54:38,401 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:54:38,401 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43727 2023-02-13 00:54:38,402 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43727 2023-02-13 00:54:38,402 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43727 2023-02-13 00:54:38,402 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43727 2023-02-13 00:54:38,403 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43727 2023-02-13 00:54:38,414 INFO [Listener at localhost/43763] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 00:54:38,415 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,415 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,415 INFO [Listener at localhost/43763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:54:38,415 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,415 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:54:38,416 INFO [Listener at localhost/43763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:54:38,417 INFO [Listener at localhost/43763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34557 2023-02-13 00:54:38,418 INFO [Listener at localhost/43763] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 00:54:38,418 DEBUG [Listener at localhost/43763] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 00:54:38,419 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,421 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,423 INFO [Listener at localhost/43763] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34557 connecting to ZooKeeper ensemble=127.0.0.1:63852 2023-02-13 00:54:38,427 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:345570x0, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:54:38,428 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:34557-0x1004b2c240b0002 connected 2023-02-13 00:54:38,428 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:54:38,429 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:54:38,429 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:54:38,430 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34557 2023-02-13 00:54:38,430 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34557 2023-02-13 00:54:38,430 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34557 2023-02-13 00:54:38,431 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34557 2023-02-13 00:54:38,431 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34557 2023-02-13 00:54:38,443 INFO [Listener at localhost/43763] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-13 00:54:38,443 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,443 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,444 INFO [Listener at localhost/43763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 00:54:38,444 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 00:54:38,444 INFO [Listener at localhost/43763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 00:54:38,444 INFO [Listener at localhost/43763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 00:54:38,445 INFO [Listener at localhost/43763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41539 2023-02-13 00:54:38,446 INFO [Listener at localhost/43763] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 00:54:38,447 DEBUG [Listener at localhost/43763] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 00:54:38,448 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,449 INFO [Listener at localhost/43763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,450 INFO [Listener at localhost/43763] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41539 connecting to ZooKeeper ensemble=127.0.0.1:63852 2023-02-13 00:54:38,454 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:415390x0, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 00:54:38,455 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41539-0x1004b2c240b0003 connected 2023-02-13 00:54:38,455 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:54:38,456 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:54:38,457 DEBUG [Listener at localhost/43763] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 00:54:38,457 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41539 2023-02-13 00:54:38,457 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41539 2023-02-13 00:54:38,460 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41539 2023-02-13 00:54:38,460 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41539 2023-02-13 00:54:38,460 DEBUG [Listener at localhost/43763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41539 2023-02-13 00:54:38,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:38,470 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 00:54:38,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:38,490 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:54:38,490 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:54:38,490 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:54:38,490 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 00:54:38,490 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:38,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 00:54:38,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43619,1676249677220 from backup master directory 2023-02-13 00:54:38,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 00:54:38,496 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:38,496 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 00:54:38,497 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:54:38,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:38,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 00:54:38,501 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 00:54:38,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/hbase.id with ID: 5060a48c-3704-44ef-ba8a-13cf94c2cbe3 2023-02-13 00:54:38,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 00:54:38,643 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:38,689 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x48e92653 to 127.0.0.1:63852 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:38,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b280bcc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:38,742 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 00:54:38,744 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 00:54:38,760 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 00:54:38,760 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 00:54:38,763 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:54:38,768 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:54:38,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:54:38,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store-tmp 2023-02-13 00:54:38,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 00:54:38,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:54:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:54:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 00:54:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:54:38,836 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:54:38,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676249678835Disabling compacts and flushes for region at 1676249678836 (+1 ms)Disabling writes for close at 1676249678836Writing region close event to WAL at 1676249678836Closed at 1676249678836 2023-02-13 00:54:38,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/WALs/jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:38,857 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43619%2C1676249677220, suffix=, logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/WALs/jenkins-hbase4.apache.org,43619,1676249677220, archiveDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/oldWALs, maxLogs=10 2023-02-13 00:54:38,910 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK] 2023-02-13 00:54:38,910 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK] 2023-02-13 00:54:38,910 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK] 2023-02-13 00:54:38,918 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:54:38,989 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/WALs/jenkins-hbase4.apache.org,43619,1676249677220/jenkins-hbase4.apache.org%2C43619%2C1676249677220.1676249678867 2023-02-13 00:54:38,989 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK], DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK], DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK]] 2023-02-13 00:54:38,990 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:38,991 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:38,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:38,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:39,055 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:39,061 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 00:54:39,085 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 00:54:39,097 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:39,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:39,104 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:39,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 00:54:39,122 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:39,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72929119, jitterRate=0.08672855794429779}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:39,123 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676249679021Initializing all the Stores at 1676249679023 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249679024 (+1 ms)Cleaning up temporary data from old regions at 1676249679110 (+86 ms)Cleaning up detritus from prior splits at 1676249679111 (+1 ms)Region opened successfully at 1676249679123 (+12 ms) 2023-02-13 00:54:39,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 00:54:39,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:39,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:39,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:39,150 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 00:54:39,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-13 00:54:39,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 00:54:39,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 00:54:39,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 00:54:39,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 00:54:39,242 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 00:54:39,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 00:54:39,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 00:54:39,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 00:54:39,256 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:39,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 00:54:39,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 00:54:39,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 00:54:39,275 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:54:39,275 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:54:39,277 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:54:39,277 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 00:54:39,278 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:39,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43619,1676249677220, sessionid=0x1004b2c240b0000, setting cluster-up flag (Was=false) 2023-02-13 00:54:39,295 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:39,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 00:54:39,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:39,310 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:39,315 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 00:54:39,316 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:39,319 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.hbase-snapshot/.tmp 2023-02-13 00:54:39,364 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(952): ClusterId : 5060a48c-3704-44ef-ba8a-13cf94c2cbe3 2023-02-13 00:54:39,364 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(952): ClusterId : 5060a48c-3704-44ef-ba8a-13cf94c2cbe3 2023-02-13 00:54:39,364 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(952): ClusterId : 5060a48c-3704-44ef-ba8a-13cf94c2cbe3 2023-02-13 00:54:39,371 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 00:54:39,371 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 00:54:39,371 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 00:54:39,378 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 00:54:39,378 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 00:54:39,378 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 00:54:39,379 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 00:54:39,379 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 00:54:39,379 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 00:54:39,382 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 00:54:39,382 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 00:54:39,384 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 00:54:39,384 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ReadOnlyZKClient(139): Connect 0x32babb26 to 127.0.0.1:63852 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:39,384 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ReadOnlyZKClient(139): Connect 0x5ceda779 to 127.0.0.1:63852 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:39,385 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ReadOnlyZKClient(139): Connect 0x64cdf83b to 127.0.0.1:63852 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:39,425 DEBUG [RS:1;jenkins-hbase4:34557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a0a1ad2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:39,425 DEBUG [RS:0;jenkins-hbase4:43727] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c7d4301, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:39,425 DEBUG [RS:1;jenkins-hbase4:34557] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e35934b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:54:39,425 DEBUG [RS:2;jenkins-hbase4:41539] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@522a4294, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:39,426 DEBUG [RS:0;jenkins-hbase4:43727] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3842d056, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:54:39,426 DEBUG [RS:2;jenkins-hbase4:41539] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5615cc09, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:54:39,448 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41539 2023-02-13 00:54:39,448 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:43727 2023-02-13 00:54:39,449 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34557 2023-02-13 00:54:39,452 INFO [RS:2;jenkins-hbase4:41539] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 00:54:39,452 INFO [RS:1;jenkins-hbase4:34557] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 00:54:39,452 INFO [RS:0;jenkins-hbase4:43727] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 00:54:39,453 INFO [RS:0;jenkins-hbase4:43727] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 00:54:39,452 INFO [RS:1;jenkins-hbase4:34557] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 00:54:39,452 INFO [RS:2;jenkins-hbase4:41539] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 00:54:39,453 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 00:54:39,453 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 00:54:39,453 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 00:54:39,456 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43619,1676249677220 with isa=jenkins-hbase4.apache.org/172.31.14.131:34557, startcode=1676249678414 2023-02-13 00:54:39,456 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43619,1676249677220 with isa=jenkins-hbase4.apache.org/172.31.14.131:43727, startcode=1676249678372 2023-02-13 00:54:39,456 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43619,1676249677220 with isa=jenkins-hbase4.apache.org/172.31.14.131:41539, startcode=1676249678442 2023-02-13 00:54:39,462 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:39,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:54:39,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:54:39,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:54:39,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-13 00:54:39,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-13 00:54:39,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,473 DEBUG [RS:2;jenkins-hbase4:41539] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 00:54:39,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:54:39,473 DEBUG [RS:0;jenkins-hbase4:43727] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 00:54:39,473 DEBUG [RS:1;jenkins-hbase4:34557] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 00:54:39,473 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249709474 2023-02-13 00:54:39,476 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 00:54:39,480 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:39,481 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 00:54:39,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 00:54:39,487 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 00:54:39,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 00:54:39,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 00:54:39,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 00:54:39,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 00:54:39,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 00:54:39,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 00:54:39,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 00:54:39,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 00:54:39,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 00:54:39,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249679510,5,FailOnTimeoutGroup] 2023-02-13 00:54:39,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249679513,5,FailOnTimeoutGroup] 2023-02-13 00:54:39,514 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,514 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 00:54:39,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,542 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:39,543 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:39,544 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac 2023-02-13 00:54:39,546 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42671, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 00:54:39,546 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39359, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 00:54:39,546 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34291, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 00:54:39,562 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43619] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,574 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:39,577 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 00:54:39,580 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/info 2023-02-13 00:54:39,581 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 00:54:39,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:39,582 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 00:54:39,586 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:54:39,586 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac 2023-02-13 00:54:39,586 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac 2023-02-13 00:54:39,586 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41999 2023-02-13 00:54:39,587 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41999 2023-02-13 00:54:39,587 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 00:54:39,587 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 00:54:39,587 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 00:54:39,588 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac 2023-02-13 00:54:39,588 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:41999 2023-02-13 00:54:39,588 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 00:54:39,588 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:39,589 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 00:54:39,592 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/table 2023-02-13 00:54:39,592 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 00:54:39,592 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:54:39,593 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:39,595 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740 2023-02-13 00:54:39,595 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740 2023-02-13 00:54:39,596 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ZKUtil(162): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,596 WARN [RS:1;jenkins-hbase4:34557] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:54:39,596 INFO [RS:1;jenkins-hbase4:34557] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:54:39,597 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,597 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ZKUtil(162): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,597 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,43727,1676249678372] 2023-02-13 00:54:39,597 WARN [RS:2;jenkins-hbase4:41539] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:54:39,597 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41539,1676249678442] 2023-02-13 00:54:39,597 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ZKUtil(162): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,598 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34557,1676249678414] 2023-02-13 00:54:39,597 INFO [RS:2;jenkins-hbase4:41539] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:54:39,598 WARN [RS:0;jenkins-hbase4:43727] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 00:54:39,599 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,599 INFO [RS:0;jenkins-hbase4:43727] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:54:39,599 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1947): logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,605 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 00:54:39,610 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 00:54:39,612 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ZKUtil(162): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,612 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ZKUtil(162): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,613 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ZKUtil(162): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,613 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ZKUtil(162): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,614 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ZKUtil(162): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,614 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ZKUtil(162): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,614 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ZKUtil(162): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,614 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ZKUtil(162): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,615 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ZKUtil(162): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,621 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:39,622 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71957383, jitterRate=0.07224856317043304}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 00:54:39,622 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676249679574Initializing all the Stores at 1676249679576 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249679576Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249679576Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249679576Cleaning up temporary data from old regions at 1676249679598 (+22 ms)Cleaning up detritus from prior splits at 1676249679600 (+2 ms)Region opened successfully at 1676249679622 (+22 ms) 2023-02-13 00:54:39,622 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 00:54:39,622 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 00:54:39,622 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 00:54:39,622 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 00:54:39,622 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 00:54:39,625 DEBUG [RS:1;jenkins-hbase4:34557] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 00:54:39,625 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 00:54:39,625 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 00:54:39,627 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 00:54:39,627 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676249679622Disabling compacts and flushes for region at 1676249679622Disabling writes for close at 1676249679622Writing region close event to WAL at 1676249679627 (+5 ms)Closed at 1676249679627 2023-02-13 00:54:39,637 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:39,637 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 00:54:39,641 INFO [RS:1;jenkins-hbase4:34557] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 00:54:39,641 INFO [RS:2;jenkins-hbase4:41539] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 00:54:39,642 INFO [RS:0;jenkins-hbase4:43727] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 00:54:39,646 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 00:54:39,660 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 00:54:39,662 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 00:54:39,665 INFO [RS:1;jenkins-hbase4:34557] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 00:54:39,665 INFO [RS:0;jenkins-hbase4:43727] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 00:54:39,665 INFO [RS:2;jenkins-hbase4:41539] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 00:54:39,670 INFO [RS:1;jenkins-hbase4:34557] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:54:39,670 INFO [RS:0;jenkins-hbase4:43727] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:54:39,670 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,670 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,670 INFO [RS:2;jenkins-hbase4:41539] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 00:54:39,671 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,671 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 00:54:39,671 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 00:54:39,671 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 00:54:39,679 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,679 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,679 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,680 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,680 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-13 00:54:39,681 DEBUG [RS:2;jenkins-hbase4:41539] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,681 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,682 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,682 DEBUG [RS:1;jenkins-hbase4:34557] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,682 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,682 DEBUG [RS:0;jenkins-hbase4:43727] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-13 00:54:39,683 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,683 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,683 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,683 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,684 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,684 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,684 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,684 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,684 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,700 INFO [RS:0;jenkins-hbase4:43727] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 00:54:39,700 INFO [RS:1;jenkins-hbase4:34557] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 00:54:39,700 INFO [RS:2;jenkins-hbase4:41539] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 00:54:39,703 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34557,1676249678414-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,703 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41539,1676249678442-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,703 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43727,1676249678372-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:39,721 INFO [RS:1;jenkins-hbase4:34557] regionserver.Replication(203): jenkins-hbase4.apache.org,34557,1676249678414 started 2023-02-13 00:54:39,721 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,34557,1676249678414, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34557, sessionid=0x1004b2c240b0002 2023-02-13 00:54:39,722 INFO [RS:0;jenkins-hbase4:43727] regionserver.Replication(203): jenkins-hbase4.apache.org,43727,1676249678372 started 2023-02-13 00:54:39,722 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,43727,1676249678372, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:43727, sessionid=0x1004b2c240b0001 2023-02-13 00:54:39,722 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 00:54:39,722 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 00:54:39,722 DEBUG [RS:1;jenkins-hbase4:34557] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,722 DEBUG [RS:0;jenkins-hbase4:43727] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,723 INFO [RS:2;jenkins-hbase4:41539] regionserver.Replication(203): jenkins-hbase4.apache.org,41539,1676249678442 started 2023-02-13 00:54:39,723 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34557,1676249678414' 2023-02-13 00:54:39,723 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41539,1676249678442, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41539, sessionid=0x1004b2c240b0003 2023-02-13 00:54:39,723 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43727,1676249678372' 2023-02-13 00:54:39,724 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 00:54:39,724 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:39,723 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:39,724 DEBUG [RS:2;jenkins-hbase4:41539] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,724 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41539,1676249678442' 2023-02-13 00:54:39,724 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:39,724 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:39,724 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:39,724 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:39,725 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 00:54:39,725 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 00:54:39,725 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 00:54:39,725 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 00:54:39,725 DEBUG [RS:1;jenkins-hbase4:34557] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:39,725 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 00:54:39,725 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34557,1676249678414' 2023-02-13 00:54:39,726 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:54:39,725 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 00:54:39,725 DEBUG [RS:2;jenkins-hbase4:41539] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:39,726 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41539,1676249678442' 2023-02-13 00:54:39,726 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:54:39,726 DEBUG [RS:0;jenkins-hbase4:43727] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:39,726 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,43727,1676249678372' 2023-02-13 00:54:39,726 DEBUG [RS:1;jenkins-hbase4:34557] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:54:39,726 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 00:54:39,727 DEBUG [RS:2;jenkins-hbase4:41539] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:54:39,727 DEBUG [RS:1;jenkins-hbase4:34557] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 00:54:39,727 DEBUG [RS:0;jenkins-hbase4:43727] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 00:54:39,727 DEBUG [RS:2;jenkins-hbase4:41539] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 00:54:39,727 INFO [RS:1;jenkins-hbase4:34557] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 00:54:39,727 INFO [RS:2;jenkins-hbase4:41539] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 00:54:39,729 INFO [RS:1;jenkins-hbase4:34557] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 00:54:39,729 DEBUG [RS:0;jenkins-hbase4:43727] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 00:54:39,729 INFO [RS:2;jenkins-hbase4:41539] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 00:54:39,729 INFO [RS:0;jenkins-hbase4:43727] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 00:54:39,729 INFO [RS:0;jenkins-hbase4:43727] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 00:54:39,815 DEBUG [jenkins-hbase4:43619] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 00:54:39,819 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:54:39,825 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:54:39,825 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:54:39,825 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:54:39,825 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:54:39,828 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43727,1676249678372, state=OPENING 2023-02-13 00:54:39,836 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 00:54:39,837 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:39,838 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 00:54:39,840 INFO [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43727%2C1676249678372, suffix=, logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,43727,1676249678372, archiveDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs, maxLogs=32 2023-02-13 00:54:39,840 INFO [RS:2;jenkins-hbase4:41539] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41539%2C1676249678442, suffix=, logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,41539,1676249678442, archiveDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs, maxLogs=32 2023-02-13 00:54:39,840 INFO [RS:1;jenkins-hbase4:34557] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34557%2C1676249678414, suffix=, logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,34557,1676249678414, archiveDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs, maxLogs=32 2023-02-13 00:54:39,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:54:39,874 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK] 2023-02-13 00:54:39,874 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK] 2023-02-13 00:54:39,874 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK] 2023-02-13 00:54:39,876 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK] 2023-02-13 00:54:39,876 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK] 2023-02-13 00:54:39,876 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK] 2023-02-13 00:54:39,876 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK] 2023-02-13 00:54:39,877 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK] 2023-02-13 00:54:39,877 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK] 2023-02-13 00:54:39,892 INFO [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,43727,1676249678372/jenkins-hbase4.apache.org%2C43727%2C1676249678372.1676249679846 2023-02-13 00:54:39,892 INFO [RS:2;jenkins-hbase4:41539] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,41539,1676249678442/jenkins-hbase4.apache.org%2C41539%2C1676249678442.1676249679845 2023-02-13 00:54:39,895 DEBUG [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK], DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK], DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK]] 2023-02-13 00:54:39,896 DEBUG [RS:2;jenkins-hbase4:41539] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK], DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK], DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK]] 2023-02-13 00:54:39,896 INFO [RS:1;jenkins-hbase4:34557] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,34557,1676249678414/jenkins-hbase4.apache.org%2C34557%2C1676249678414.1676249679844 2023-02-13 00:54:39,896 DEBUG [RS:1;jenkins-hbase4:34557] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK], DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK], DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK]] 2023-02-13 00:54:40,029 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:40,032 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:54:40,035 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56552, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:54:40,047 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 00:54:40,047 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 00:54:40,051 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43727%2C1676249678372.meta, suffix=.meta, logDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,43727,1676249678372, archiveDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs, maxLogs=32 2023-02-13 00:54:40,074 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK] 2023-02-13 00:54:40,074 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK] 2023-02-13 00:54:40,076 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK] 2023-02-13 00:54:40,081 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/WALs/jenkins-hbase4.apache.org,43727,1676249678372/jenkins-hbase4.apache.org%2C43727%2C1676249678372.meta.1676249680053.meta 2023-02-13 00:54:40,081 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:36179,DS-5925e93f-f8d6-4a5a-90e2-a6a14078def5,DISK], DatanodeInfoWithStorage[127.0.0.1:38789,DS-56a297bc-d869-4093-96c2-e30c2af4c59b,DISK], DatanodeInfoWithStorage[127.0.0.1:44837,DS-e3e9eab9-42b0-48f8-8d61-0cee93f0da69,DISK]] 2023-02-13 00:54:40,081 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:40,086 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:40,086 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 00:54:40,102 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 00:54:40,104 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 00:54:40,108 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 00:54:40,109 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:40,109 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 00:54:40,109 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 00:54:40,111 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 00:54:40,113 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/info 2023-02-13 00:54:40,113 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/info 2023-02-13 00:54:40,114 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 00:54:40,115 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:40,115 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 00:54:40,116 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:54:40,116 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/rep_barrier 2023-02-13 00:54:40,117 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 00:54:40,118 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:40,118 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 00:54:40,119 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/table 2023-02-13 00:54:40,119 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/table 2023-02-13 00:54:40,120 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 00:54:40,120 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:40,122 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740 2023-02-13 00:54:40,127 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740 2023-02-13 00:54:40,132 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 00:54:40,134 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 00:54:40,135 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72895217, jitterRate=0.08622337877750397}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 00:54:40,135 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676249680109Writing region info on filesystem at 1676249680109Initializing all the Stores at 1676249680111 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249680111Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249680111Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249680111Cleaning up temporary data from old regions at 1676249680130 (+19 ms)Cleaning up detritus from prior splits at 1676249680130Running coprocessor post-open hooks at 1676249680135 (+5 ms)Region opened successfully at 1676249680135 2023-02-13 00:54:40,146 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676249680019 2023-02-13 00:54:40,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 00:54:40,163 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 00:54:40,164 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,43727,1676249678372, state=OPEN 2023-02-13 00:54:40,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 00:54:40,168 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 00:54:40,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 00:54:40,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,43727,1676249678372 in 326 msec 2023-02-13 00:54:40,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 00:54:40,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 528 msec 2023-02-13 00:54:40,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 820 msec 2023-02-13 00:54:40,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676249678485, completionTime=-1 2023-02-13 00:54:40,183 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 00:54:40,183 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:40,254 DEBUG [hconnection-0x5c02c0c4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:40,257 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56568, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:40,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:40,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249740275 2023-02-13 00:54:40,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249800275 2023-02-13 00:54:40,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 91 msec 2023-02-13 00:54:40,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43619,1676249677220-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:40,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43619,1676249677220-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:40,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43619,1676249677220-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:40,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43619, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:40,302 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 00:54:40,312 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 00:54:40,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 00:54:40,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 00:54:40,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:40,336 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:40,339 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:40,361 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,364 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d empty. 2023-02-13 00:54:40,364 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,364 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 00:54:40,403 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:40,406 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2a8283ab9ffc3d4f3875786a1eb0696d, NAME => 'hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 2a8283ab9ffc3d4f3875786a1eb0696d, disabling compactions & flushes 2023-02-13 00:54:40,427 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. after waiting 0 ms 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,427 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,427 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 2a8283ab9ffc3d4f3875786a1eb0696d: Waiting for close lock at 1676249680427Disabling compacts and flushes for region at 1676249680427Disabling writes for close at 1676249680427Writing region close event to WAL at 1676249680427Closed at 1676249680427 2023-02-13 00:54:40,433 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:40,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249680436"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249680436"}]},"ts":"1676249680436"} 2023-02-13 00:54:40,476 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:40,478 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:40,483 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249680479"}]},"ts":"1676249680479"} 2023-02-13 00:54:40,487 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 00:54:40,492 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:54:40,493 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:54:40,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:54:40,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:54:40,494 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:54:40,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2a8283ab9ffc3d4f3875786a1eb0696d, ASSIGN}] 2023-02-13 00:54:40,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=2a8283ab9ffc3d4f3875786a1eb0696d, ASSIGN 2023-02-13 00:54:40,502 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=2a8283ab9ffc3d4f3875786a1eb0696d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:40,654 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:54:40,655 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:40,656 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249680655"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249680655"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249680655"}]},"ts":"1676249680655"} 2023-02-13 00:54:40,661 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 2a8283ab9ffc3d4f3875786a1eb0696d, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:40,816 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:40,816 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:54:40,821 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40394, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:54:40,828 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,830 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2a8283ab9ffc3d4f3875786a1eb0696d, NAME => 'hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:40,831 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:40,831 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,831 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:40,831 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,832 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,834 INFO [StoreOpener-2a8283ab9ffc3d4f3875786a1eb0696d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,836 DEBUG [StoreOpener-2a8283ab9ffc3d4f3875786a1eb0696d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/info 2023-02-13 00:54:40,836 DEBUG [StoreOpener-2a8283ab9ffc3d4f3875786a1eb0696d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/info 2023-02-13 00:54:40,836 INFO [StoreOpener-2a8283ab9ffc3d4f3875786a1eb0696d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a8283ab9ffc3d4f3875786a1eb0696d columnFamilyName info 2023-02-13 00:54:40,837 INFO [StoreOpener-2a8283ab9ffc3d4f3875786a1eb0696d-1] regionserver.HStore(310): Store=2a8283ab9ffc3d4f3875786a1eb0696d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:40,839 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,840 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,844 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:54:40,848 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:40,849 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2a8283ab9ffc3d4f3875786a1eb0696d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64277666, jitterRate=-0.0421881377696991}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:54:40,849 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2a8283ab9ffc3d4f3875786a1eb0696d: Running coprocessor pre-open hook at 1676249680832Writing region info on filesystem at 1676249680832Initializing all the Stores at 1676249680833 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676249680833Cleaning up temporary data from old regions at 1676249680841 (+8 ms)Cleaning up detritus from prior splits at 1676249680841Running coprocessor post-open hooks at 1676249680849 (+8 ms)Region opened successfully at 1676249680849 2023-02-13 00:54:40,851 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d., pid=6, masterSystemTime=1676249680816 2023-02-13 00:54:40,856 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,856 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:54:40,858 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:40,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676249680857"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249680857"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249680857"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249680857"}]},"ts":"1676249680857"} 2023-02-13 00:54:40,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 00:54:40,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 2a8283ab9ffc3d4f3875786a1eb0696d, server=jenkins-hbase4.apache.org,41539,1676249678442 in 202 msec 2023-02-13 00:54:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 00:54:40,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=2a8283ab9ffc3d4f3875786a1eb0696d, ASSIGN in 369 msec 2023-02-13 00:54:40,872 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:40,872 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249680872"}]},"ts":"1676249680872"} 2023-02-13 00:54:40,875 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 00:54:40,878 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:40,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 554 msec 2023-02-13 00:54:40,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 00:54:40,943 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:54:40,943 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:40,965 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:40,967 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40398, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:40,981 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:41,000 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:54:41,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 40 msec 2023-02-13 00:54:41,024 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:41,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 00:54:41,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-13 00:54:41,051 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 00:54:41,054 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 00:54:41,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.557sec 2023-02-13 00:54:41,057 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 00:54:41,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 00:54:41,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 00:54:41,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43619,1676249677220-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 00:54:41,062 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43619,1676249677220-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 00:54:41,070 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 00:54:41,070 DEBUG [Listener at localhost/43763] zookeeper.ReadOnlyZKClient(139): Connect 0x328d140f to 127.0.0.1:63852 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 00:54:41,075 DEBUG [Listener at localhost/43763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37e887de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 00:54:41,090 DEBUG [hconnection-0x5adffcfe-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:41,101 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56580, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:41,111 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:54:41,141 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=358, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=216, ProcessCount=168, AvailableMemoryMB=3832 2023-02-13 00:54:41,146 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:41,156 DEBUG [Listener at localhost/43763] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 00:54:41,160 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58626, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 00:54:41,173 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:54:41,174 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:54:41,175 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:54:41,180 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:41,182 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:41,184 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:41,187 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,188 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf empty. 2023-02-13 00:54:41,191 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 00:54:41,212 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:41,214 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => f8ce95eb513c26321825effddeecdacf, NAME => 'testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing f8ce95eb513c26321825effddeecdacf, disabling compactions & flushes 2023-02-13 00:54:41,639 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. after waiting 0 ms 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:41,639 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:41,639 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for f8ce95eb513c26321825effddeecdacf: Waiting for close lock at 1676249681639Disabling compacts and flushes for region at 1676249681639Disabling writes for close at 1676249681639Writing region close event to WAL at 1676249681639Closed at 1676249681639 2023-02-13 00:54:41,645 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:41,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249681647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249681647"}]},"ts":"1676249681647"} 2023-02-13 00:54:41,650 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:41,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:41,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249681652"}]},"ts":"1676249681652"} 2023-02-13 00:54:41,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 00:54:41,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:54:41,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:54:41,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:54:41,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:54:41,662 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:54:41,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, ASSIGN}] 2023-02-13 00:54:41,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, ASSIGN 2023-02-13 00:54:41,668 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:41,818 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:54:41,819 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f8ce95eb513c26321825effddeecdacf, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:41,819 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249681819"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249681819"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249681819"}]},"ts":"1676249681819"} 2023-02-13 00:54:41,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f8ce95eb513c26321825effddeecdacf, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:41,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:41,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f8ce95eb513c26321825effddeecdacf, NAME => 'testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:41,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:41,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:41,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,985 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,988 DEBUG [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1 2023-02-13 00:54:41,988 DEBUG [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1 2023-02-13 00:54:41,988 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f8ce95eb513c26321825effddeecdacf columnFamilyName cf1 2023-02-13 00:54:41,989 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] regionserver.HStore(310): Store=f8ce95eb513c26321825effddeecdacf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:41,989 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,992 DEBUG [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2 2023-02-13 00:54:41,992 DEBUG [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2 2023-02-13 00:54:41,992 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f8ce95eb513c26321825effddeecdacf columnFamilyName cf2 2023-02-13 00:54:41,993 INFO [StoreOpener-f8ce95eb513c26321825effddeecdacf-1] regionserver.HStore(310): Store=f8ce95eb513c26321825effddeecdacf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:41,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:41,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:42,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:42,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:42,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:42,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f8ce95eb513c26321825effddeecdacf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68331424, jitterRate=0.01821756362915039}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:42,008 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f8ce95eb513c26321825effddeecdacf: Running coprocessor pre-open hook at 1676249681983Writing region info on filesystem at 1676249681983Initializing all the Stores at 1676249681985 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249681985Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249681985Cleaning up temporary data from old regions at 1676249681998 (+13 ms)Cleaning up detritus from prior splits at 1676249681999 (+1 ms)Running coprocessor post-open hooks at 1676249682007 (+8 ms)Region opened successfully at 1676249682008 (+1 ms) 2023-02-13 00:54:42,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf., pid=11, masterSystemTime=1676249681975 2023-02-13 00:54:42,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,013 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f8ce95eb513c26321825effddeecdacf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,014 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249682013"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249682013"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249682013"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249682013"}]},"ts":"1676249682013"} 2023-02-13 00:54:42,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 00:54:42,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f8ce95eb513c26321825effddeecdacf, server=jenkins-hbase4.apache.org,41539,1676249678442 in 195 msec 2023-02-13 00:54:42,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 00:54:42,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, ASSIGN in 360 msec 2023-02-13 00:54:42,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:42,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249682026"}]},"ts":"1676249682026"} 2023-02-13 00:54:42,029 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 00:54:42,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:42,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 854 msec 2023-02-13 00:54:42,186 DEBUG [Listener at localhost/43763] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:54:42,191 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40408, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:54:42,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:42,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,219 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,220 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,220 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:42,220 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:42,221 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,221 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,222 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,222 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,222 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,223 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,223 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,224 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:42,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,225 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,227 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,227 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,228 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,229 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:42,230 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:42,230 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,231 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:42,233 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:54:42,233 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,233 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,233 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,234 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,234 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,234 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,326 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/ac7fb191def14c40ad6263b54bbc4e59 2023-02-13 00:54:42,404 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/2c92faeb7bd740bb9664fbf0eae63a8d 2023-02-13 00:54:42,418 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/ac7fb191def14c40ad6263b54bbc4e59 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59 2023-02-13 00:54:42,429 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:42,433 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/2c92faeb7bd740bb9664fbf0eae63a8d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d 2023-02-13 00:54:42,447 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:42,449 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 216ms, sequenceid=6, compaction requested=false 2023-02-13 00:54:42,451 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,453 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:42,453 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,454 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:42,456 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:42,456 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,457 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,457 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,460 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,461 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,461 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,461 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:42,467 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:42,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e8d80e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:42,467 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,472 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,472 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,472 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,473 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:42,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:42,473 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:42,476 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,476 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,476 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,476 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,476 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,477 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,478 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,482 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:42,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:42,491 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:42,491 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:42,491 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:42,491 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,492 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,497 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,498 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,499 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,500 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,503 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,503 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,504 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,521 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,523 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:42,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,526 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:42,527 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:42,627 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:42,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:42,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:42,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:42,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,663 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,663 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,664 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:42,664 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:42,664 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,664 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,666 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,666 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,666 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,666 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,667 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,667 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,669 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,669 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,669 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,669 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,670 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:42,671 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:42,672 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,672 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:42,672 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:42,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,675 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,675 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,696 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec 2023-02-13 00:54:42,722 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/5a6e9301366f47ca91ac0030d7c8c742 2023-02-13 00:54:42,731 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec 2023-02-13 00:54:42,739 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:42,740 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/5a6e9301366f47ca91ac0030d7c8c742 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742 2023-02-13 00:54:42,748 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:42,750 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 78ms, sequenceid=19, compaction requested=false 2023-02-13 00:54:42,750 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:42,750 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,750 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:42,751 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:42,751 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,751 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,751 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,753 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,754 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,754 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:42,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@373b2b18[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:42,756 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:42,757 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,759 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,759 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,759 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,759 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:42,759 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,759 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,760 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,760 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:42,760 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:42,760 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,761 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,762 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,763 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:42,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:42,768 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:42,768 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:42,768 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:42,768 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,768 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:42,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,772 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,771 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:42,772 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,773 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,776 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:42,776 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,777 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,787 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:42,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,787 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,788 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:42,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:42,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,790 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:42,790 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:42,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:42,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:42,890 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:42,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:42,917 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:42,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:42,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,919 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:42,919 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,920 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:42,920 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:42,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,922 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,923 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:42,924 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:42,925 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,925 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,925 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,926 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,926 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,926 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:42,928 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,926 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,929 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:42,929 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:42,934 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,935 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,935 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,935 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,935 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:42,935 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:42,948 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/132cf935ad694ac59f53608775ff97cb 2023-02-13 00:54:42,972 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/16bff9c2ffe24e348c04433ef8a8b67f 2023-02-13 00:54:42,981 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/132cf935ad694ac59f53608775ff97cb as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb 2023-02-13 00:54:42,991 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:42,993 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/16bff9c2ffe24e348c04433ef8a8b67f as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f 2023-02-13 00:54:43,002 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:43,004 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 75ms, sequenceid=32, compaction requested=true 2023-02-13 00:54:43,004 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:43,004 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:43,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:43,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,007 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@726260da[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:43,010 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:43,010 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,012 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,011 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,011 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,012 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,012 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:43,012 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:43,012 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:43,012 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:43,015 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,015 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,015 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,016 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,016 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,016 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,016 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,017 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,017 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:43,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:43,022 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:43,022 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:43,022 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:43,022 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,022 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,024 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,025 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,025 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,025 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,029 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,031 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,031 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,031 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,036 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,037 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:43,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,036 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,036 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,037 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,040 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,040 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:43,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,041 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,141 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:43,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:43,166 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:43,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:43,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,169 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,170 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:43,170 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:43,170 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,170 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,172 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,172 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,172 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,173 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,173 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,175 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,177 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,177 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,176 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:43,178 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,178 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:43,178 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:43,178 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:43,179 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,180 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,180 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,180 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,180 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,180 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,199 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/6338d37bc23942b69325458d6326a1d7 2023-02-13 00:54:43,227 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/4845d941c0d04c698215c80981b45fbc 2023-02-13 00:54:43,236 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/6338d37bc23942b69325458d6326a1d7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7 2023-02-13 00:54:43,244 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:43,245 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/4845d941c0d04c698215c80981b45fbc as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc 2023-02-13 00:54:43,253 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:43,254 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 76ms, sequenceid=45, compaction requested=true 2023-02-13 00:54:43,255 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:43,255 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,255 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:43,256 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:43,256 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,256 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,256 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,261 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,261 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,262 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,262 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:43,266 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c17bc09[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:43,266 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:43,266 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,268 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,268 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,271 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,271 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,271 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,271 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,271 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,272 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,272 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:43,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:43,276 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:43,277 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:43,277 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:43,277 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,277 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,280 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,280 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,281 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,282 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,284 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,284 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,285 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,288 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,288 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,288 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,289 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:43,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,289 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,290 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,390 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:43,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:43,413 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:43,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:43,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,417 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,417 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:43,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:43,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,419 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,420 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,419 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,422 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,422 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,422 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,425 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,427 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,427 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,427 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:43,428 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:43,429 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,430 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,430 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,430 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,430 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,430 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,451 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/409a1062d8dc4551b974cbbf35a3e205 2023-02-13 00:54:43,478 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/5f42ebd1779c4cdca34e858e600f28c4 2023-02-13 00:54:43,490 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/409a1062d8dc4551b974cbbf35a3e205 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205 2023-02-13 00:54:43,497 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:43,498 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/5f42ebd1779c4cdca34e858e600f28c4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4 2023-02-13 00:54:43,506 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:43,507 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 79ms, sequenceid=58, compaction requested=true 2023-02-13 00:54:43,507 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:43,507 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,507 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:43,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:43,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,514 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,514 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,515 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,515 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d7e2b2d[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:43,517 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:43,517 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,518 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:43,519 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,521 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,522 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,522 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,522 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,522 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,522 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,523 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:43,528 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:43,528 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:43,528 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:43,528 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:43,528 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,528 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,532 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,531 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,532 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,533 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,533 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,535 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,535 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,536 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,541 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,541 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,542 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:43,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,542 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,541 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,542 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,542 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:43,542 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,542 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:43,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,543 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,543 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,543 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,543 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,543 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,643 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:43,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:43,666 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:43,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:43,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,668 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,668 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,669 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 00:54:43,669 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 00:54:43,670 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,670 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,671 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,671 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,672 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:43,673 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,673 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,673 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,673 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,673 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,673 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,673 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,674 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:43,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,675 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 00:54:43,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,676 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,676 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:43,676 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:43,676 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,676 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. started... 2023-02-13 00:54:43,676 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:43,678 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,678 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,678 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,679 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,679 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,679 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,712 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/d64157c92491406bba607631996d473f 2023-02-13 00:54:43,742 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/b83ee500360943f999cb1befdaa653f9 2023-02-13 00:54:43,754 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/d64157c92491406bba607631996d473f as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f 2023-02-13 00:54:43,762 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:43,763 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/b83ee500360943f999cb1befdaa653f9 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9 2023-02-13 00:54:43,771 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:43,772 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 96ms, sequenceid=71, compaction requested=true 2023-02-13 00:54:43,772 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f8ce95eb513c26321825effddeecdacf: 2023-02-13 00:54:43,773 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:43,773 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 00:54:43,774 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 00:54:43,776 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,776 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,776 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,776 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:43,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 00:54:43,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13f3bc71[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:43,778 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 00:54:43,778 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:43,780 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,780 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 00:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 00:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,784 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,784 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,785 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,784 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 00:54:43,785 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,785 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,785 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 00:54:43,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:43,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:54:43,790 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 00:54:43,790 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:43,790 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:43,790 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,790 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,793 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:43,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,793 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:43,796 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:43,798 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,798 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,799 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,802 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,802 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 00:54:43,802 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:43,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:43,802 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:43,802 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,802 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,802 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:43,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:43,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:43,803 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:43,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,804 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:43,804 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 00:54:43,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:43,903 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:43,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:43,926 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,932 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:43,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:43,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:43,933 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:43,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, UNASSIGN}] 2023-02-13 00:54:43,942 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, UNASSIGN 2023-02-13 00:54:43,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f8ce95eb513c26321825effddeecdacf, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:43,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249683943"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249683943"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249683943"}]},"ts":"1676249683943"} 2023-02-13 00:54:43,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure f8ce95eb513c26321825effddeecdacf, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:44,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f8ce95eb513c26321825effddeecdacf, disabling compactions & flushes 2023-02-13 00:54:44,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:44,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:44,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. after waiting 0 ms 2023-02-13 00:54:44,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:44,105 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing f8ce95eb513c26321825effddeecdacf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:54:44,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/c58ca86aa3d04a58bbf21f24a6545be6 2023-02-13 00:54:44,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/f91bf14383a249c1b46d658e882aaee1 2023-02-13 00:54:44,154 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf1/c58ca86aa3d04a58bbf21f24a6545be6 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6 2023-02-13 00:54:44,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:44,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.tmp/cf2/f91bf14383a249c1b46d658e882aaee1 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1 2023-02-13 00:54:44,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:44,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 66ms, sequenceid=82, compaction requested=true 2023-02-13 00:54:44,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:54:44,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:44,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. 2023-02-13 00:54:44,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f8ce95eb513c26321825effddeecdacf: Waiting for close lock at 1676249684105Running coprocessor pre-close hooks at 1676249684105Disabling compacts and flushes for region at 1676249684105Disabling writes for close at 1676249684105Obtaining lock to block concurrent updates at 1676249684105Preparing flush snapshotting stores in f8ce95eb513c26321825effddeecdacf at 1676249684105Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249684106 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. at 1676249684106Flushing f8ce95eb513c26321825effddeecdacf/cf1: creating writer at 1676249684107 (+1 ms)Flushing f8ce95eb513c26321825effddeecdacf/cf1: appending metadata at 1676249684110 (+3 ms)Flushing f8ce95eb513c26321825effddeecdacf/cf1: closing flushed file at 1676249684110Flushing f8ce95eb513c26321825effddeecdacf/cf2: creating writer at 1676249684130 (+20 ms)Flushing f8ce95eb513c26321825effddeecdacf/cf2: appending metadata at 1676249684133 (+3 ms)Flushing f8ce95eb513c26321825effddeecdacf/cf2: closing flushed file at 1676249684133Flushing f8ce95eb513c26321825effddeecdacf/cf1: reopening flushed file at 1676249684154 (+21 ms)Flushing f8ce95eb513c26321825effddeecdacf/cf2: reopening flushed file at 1676249684164 (+10 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f8ce95eb513c26321825effddeecdacf in 66ms, sequenceid=82, compaction requested=true at 1676249684171 (+7 ms)Writing region close event to WAL at 1676249684187 (+16 ms)Running coprocessor post-close hooks at 1676249684192 (+5 ms)Closed at 1676249684193 (+1 ms) 2023-02-13 00:54:44,195 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,196 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f8ce95eb513c26321825effddeecdacf, regionState=CLOSED 2023-02-13 00:54:44,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249684196"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249684196"}]},"ts":"1676249684196"} 2023-02-13 00:54:44,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 00:54:44,201 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure f8ce95eb513c26321825effddeecdacf, server=jenkins-hbase4.apache.org,41539,1676249678442 in 254 msec 2023-02-13 00:54:44,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 00:54:44,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f8ce95eb513c26321825effddeecdacf, UNASSIGN in 262 msec 2023-02-13 00:54:44,223 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=f8ce95eb513c26321825effddeecdacf, threads=14 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,224 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,225 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,225 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,225 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,225 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,225 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,226 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,226 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,253 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f, top=true 2023-02-13 00:54:44,273 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205, top=true 2023-02-13 00:54:44,297 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4, top=true 2023-02-13 00:54:44,309 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,309 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,310 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1, top=true 2023-02-13 00:54:44,340 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205 for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,340 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,340 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4 for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,345 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,361 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1 for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,361 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,363 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9, top=true 2023-02-13 00:54:44,363 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6, top=true 2023-02-13 00:54:44,452 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9 for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,452 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,454 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/.splits/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6 for child: cd0b9cc8d658dd293604b3f54617114c, parent: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,454 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,500 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,502 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,502 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,505 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,505 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,506 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,508 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59 for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,899 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d for region: f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:44,900 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region f8ce95eb513c26321825effddeecdacf Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:54:44,951 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:44,953 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:44,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676249684956"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676249684956"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676249684956"}]},"ts":"1676249684956"} 2023-02-13 00:54:44,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249684956"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249684956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249684956"}]},"ts":"1676249684956"} 2023-02-13 00:54:44,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249684956"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249684956"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249684956"}]},"ts":"1676249684956"} 2023-02-13 00:54:45,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, ASSIGN}] 2023-02-13 00:54:45,006 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, ASSIGN 2023-02-13 00:54:45,007 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:45,008 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, ASSIGN 2023-02-13 00:54:45,009 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:45,157 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:54:45,159 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=cd0b9cc8d658dd293604b3f54617114c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,159 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685159"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249685159"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249685159"}]},"ts":"1676249685159"} 2023-02-13 00:54:45,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure cd0b9cc8d658dd293604b3f54617114c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:45,165 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fb3cb2309d059a71828c00de44bd7e7c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,166 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685165"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249685165"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249685165"}]},"ts":"1676249685165"} 2023-02-13 00:54:45,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure fb3cb2309d059a71828c00de44bd7e7c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:45,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fb3cb2309d059a71828c00de44bd7e7c, NAME => 'testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 00:54:45,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:45,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:45,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,322 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,325 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1 2023-02-13 00:54:45,325 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1 2023-02-13 00:54:45,325 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3cb2309d059a71828c00de44bd7e7c columnFamilyName cf1 2023-02-13 00:54:45,361 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-bottom 2023-02-13 00:54:45,369 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-bottom 2023-02-13 00:54:45,377 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-bottom 2023-02-13 00:54:45,385 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-bottom 2023-02-13 00:54:45,385 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(310): Store=fb3cb2309d059a71828c00de44bd7e7c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:45,386 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,387 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2 2023-02-13 00:54:45,388 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2 2023-02-13 00:54:45,388 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3cb2309d059a71828c00de44bd7e7c columnFamilyName cf2 2023-02-13 00:54:45,402 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-bottom 2023-02-13 00:54:45,408 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-bottom 2023-02-13 00:54:45,413 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-bottom 2023-02-13 00:54:45,419 DEBUG [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-bottom 2023-02-13 00:54:45,419 INFO [StoreOpener-fb3cb2309d059a71828c00de44bd7e7c-1] regionserver.HStore(310): Store=fb3cb2309d059a71828c00de44bd7e7c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:45,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:45,427 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:45,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fb3cb2309d059a71828c00de44bd7e7c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73481358, jitterRate=0.09495756030082703}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:45,428 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fb3cb2309d059a71828c00de44bd7e7c: Running coprocessor pre-open hook at 1676249685321Writing region info on filesystem at 1676249685321Initializing all the Stores at 1676249685322 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249685322Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249685322Cleaning up temporary data from old regions at 1676249685422 (+100 ms)Cleaning up detritus from prior splits at 1676249685423 (+1 ms)Running coprocessor post-open hooks at 1676249685427 (+4 ms)Region opened successfully at 1676249685428 (+1 ms) 2023-02-13 00:54:45,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c., pid=18, masterSystemTime=1676249685314 2023-02-13 00:54:45,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:54:45,431 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:54:45,431 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 00:54:45,434 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,434 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,434 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): fb3cb2309d059a71828c00de44bd7e7c/cf2 is initiating minor compaction (all files) 2023-02-13 00:54:45,434 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): fb3cb2309d059a71828c00de44bd7e7c/cf1 is initiating minor compaction (all files) 2023-02-13 00:54:45,434 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fb3cb2309d059a71828c00de44bd7e7c/cf2 in testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,434 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fb3cb2309d059a71828c00de44bd7e7c/cf1 in testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,435 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/.tmp, totalSize=19.9 K 2023-02-13 00:54:45,435 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/.tmp, totalSize=19.9 K 2023-02-13 00:54:45,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:45,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cd0b9cc8d658dd293604b3f54617114c, NAME => 'testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 00:54:45,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:45,437 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249682191 2023-02-13 00:54:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,437 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249682191 2023-02-13 00:54:45,438 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249682635 2023-02-13 00:54:45,438 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=fb3cb2309d059a71828c00de44bd7e7c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,438 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685438"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249685438"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249685438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249685438"}]},"ts":"1676249685438"} 2023-02-13 00:54:45,439 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249682892 2023-02-13 00:54:45,439 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249682635 2023-02-13 00:54:45,440 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249683144 2023-02-13 00:54:45,440 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249682892 2023-02-13 00:54:45,441 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249683144 2023-02-13 00:54:45,445 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-13 00:54:45,445 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure fb3cb2309d059a71828c00de44bd7e7c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 274 msec 2023-02-13 00:54:45,447 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1 2023-02-13 00:54:45,447 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1 2023-02-13 00:54:45,447 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd0b9cc8d658dd293604b3f54617114c columnFamilyName cf1 2023-02-13 00:54:45,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, ASSIGN in 441 msec 2023-02-13 00:54:45,464 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): fb3cb2309d059a71828c00de44bd7e7c#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:45,466 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-top 2023-02-13 00:54:45,470 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): fb3cb2309d059a71828c00de44bd7e7c#cf1#compaction#15 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:45,473 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-top 2023-02-13 00:54:45,481 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-top 2023-02-13 00:54:45,486 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-top 2023-02-13 00:54:45,507 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205 2023-02-13 00:54:45,520 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/.tmp/cf2/e447649f67cf48e8a0bdbfc127f84fe4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/e447649f67cf48e8a0bdbfc127f84fe4 2023-02-13 00:54:45,522 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/.tmp/cf1/ea95b296150345d8af3b5eb00dfa3f0b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ea95b296150345d8af3b5eb00dfa3f0b 2023-02-13 00:54:45,526 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6 2023-02-13 00:54:45,537 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f 2023-02-13 00:54:45,537 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(310): Store=cd0b9cc8d658dd293604b3f54617114c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:45,537 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,539 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2 2023-02-13 00:54:45,539 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2 2023-02-13 00:54:45,540 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cd0b9cc8d658dd293604b3f54617114c columnFamilyName cf2 2023-02-13 00:54:45,542 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fb3cb2309d059a71828c00de44bd7e7c/cf2 of fb3cb2309d059a71828c00de44bd7e7c into e447649f67cf48e8a0bdbfc127f84fe4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:45,542 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in fb3cb2309d059a71828c00de44bd7e7c/cf1 of fb3cb2309d059a71828c00de44bd7e7c into ea95b296150345d8af3b5eb00dfa3f0b(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:45,546 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fb3cb2309d059a71828c00de44bd7e7c: 2023-02-13 00:54:45,546 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fb3cb2309d059a71828c00de44bd7e7c: 2023-02-13 00:54:45,546 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c., storeName=fb3cb2309d059a71828c00de44bd7e7c/cf1, priority=12, startTime=1676249685430; duration=0sec 2023-02-13 00:54:45,546 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c., storeName=fb3cb2309d059a71828c00de44bd7e7c/cf2, priority=12, startTime=1676249685431; duration=0sec 2023-02-13 00:54:45,547 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,547 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,558 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-top 2023-02-13 00:54:45,562 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-top 2023-02-13 00:54:45,567 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-top 2023-02-13 00:54:45,572 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-top 2023-02-13 00:54:45,576 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4 2023-02-13 00:54:45,581 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9 2023-02-13 00:54:45,584 DEBUG [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1 2023-02-13 00:54:45,585 INFO [StoreOpener-cd0b9cc8d658dd293604b3f54617114c-1] regionserver.HStore(310): Store=cd0b9cc8d658dd293604b3f54617114c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:45,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:45,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:45,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cd0b9cc8d658dd293604b3f54617114c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66074668, jitterRate=-0.01541072130203247}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:45,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cd0b9cc8d658dd293604b3f54617114c: Running coprocessor pre-open hook at 1676249685438Writing region info on filesystem at 1676249685438Initializing all the Stores at 1676249685440 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249685440Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249685443 (+3 ms)Cleaning up temporary data from old regions at 1676249685588 (+145 ms)Cleaning up detritus from prior splits at 1676249685588Running coprocessor post-open hooks at 1676249685592 (+4 ms)Region opened successfully at 1676249685592 2023-02-13 00:54:45,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c., pid=17, masterSystemTime=1676249685314 2023-02-13 00:54:45,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,594 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:45,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:54:45,596 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:45,596 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,596 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): cd0b9cc8d658dd293604b3f54617114c/cf1 is initiating minor compaction (all files) 2023-02-13 00:54:45,596 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,596 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cd0b9cc8d658dd293604b3f54617114c/cf1 in testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,596 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): cd0b9cc8d658dd293604b3f54617114c/cf2 is initiating minor compaction (all files) 2023-02-13 00:54:45,596 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of cd0b9cc8d658dd293604b3f54617114c/cf2 in testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,596 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/.tmp, totalSize=34.9 K 2023-02-13 00:54:45,597 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/.tmp, totalSize=34.9 K 2023-02-13 00:54:45,597 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249682191 2023-02-13 00:54:45,597 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249682191 2023-02-13 00:54:45,598 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249682635 2023-02-13 00:54:45,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:45,598 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249682635 2023-02-13 00:54:45,599 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=cd0b9cc8d658dd293604b3f54617114c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,599 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249682892 2023-02-13 00:54:45,599 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249682892 2023-02-13 00:54:45,599 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685599"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249685599"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249685599"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249685599"}]},"ts":"1676249685599"} 2023-02-13 00:54:45,600 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249683144 2023-02-13 00:54:45,600 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249683144 2023-02-13 00:54:45,601 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249683393 2023-02-13 00:54:45,601 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249683393 2023-02-13 00:54:45,602 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249683645 2023-02-13 00:54:45,602 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249683906 2023-02-13 00:54:45,602 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249683645 2023-02-13 00:54:45,603 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249683906 2023-02-13 00:54:45,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-13 00:54:45,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure cd0b9cc8d658dd293604b3f54617114c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 441 msec 2023-02-13 00:54:45,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-13 00:54:45,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, ASSIGN in 601 msec 2023-02-13 00:54:45,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c in 1.6830 sec 2023-02-13 00:54:45,629 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): cd0b9cc8d658dd293604b3f54617114c#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:45,630 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): cd0b9cc8d658dd293604b3f54617114c#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:45,635 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:45,658 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/.tmp/cf2/19c48c975dab490ab1b6df27f923fa15 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/19c48c975dab490ab1b6df27f923fa15 2023-02-13 00:54:45,660 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/.tmp/cf1/d746d17c2a674a0997eb324fbd08fbd4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/d746d17c2a674a0997eb324fbd08fbd4 2023-02-13 00:54:45,669 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cd0b9cc8d658dd293604b3f54617114c/cf2 of cd0b9cc8d658dd293604b3f54617114c into 19c48c975dab490ab1b6df27f923fa15(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:45,669 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cd0b9cc8d658dd293604b3f54617114c/cf1 of cd0b9cc8d658dd293604b3f54617114c into d746d17c2a674a0997eb324fbd08fbd4(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:45,669 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for cd0b9cc8d658dd293604b3f54617114c: 2023-02-13 00:54:45,669 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cd0b9cc8d658dd293604b3f54617114c: 2023-02-13 00:54:45,669 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c., storeName=cd0b9cc8d658dd293604b3f54617114c/cf2, priority=9, startTime=1676249685594; duration=0sec 2023-02-13 00:54:45,669 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c., storeName=cd0b9cc8d658dd293604b3f54617114c/cf1, priority=9, startTime=1676249685593; duration=0sec 2023-02-13 00:54:45,670 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,670 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:45,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41539] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40408 deadline: 1676249745685, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf. is not online on jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,701 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 00:54:45,702 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 00:54:45,819 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 00:54:45,829 INFO [Listener at localhost/43763] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 00:54:45,832 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 00:54:45,864 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:45,873 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 00:54:45,877 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-13 00:54:45,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:45,888 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249685888"}]},"ts":"1676249685888"} 2023-02-13 00:54:45,890 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 00:54:45,893 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 00:54:45,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, UNASSIGN}] 2023-02-13 00:54:45,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:45,898 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, UNASSIGN 2023-02-13 00:54:45,899 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=cd0b9cc8d658dd293604b3f54617114c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,899 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685899"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249685899"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249685899"}]},"ts":"1676249685899"} 2023-02-13 00:54:45,901 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure cd0b9cc8d658dd293604b3f54617114c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:45,905 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, UNASSIGN 2023-02-13 00:54:45,906 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fb3cb2309d059a71828c00de44bd7e7c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:45,906 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249685906"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249685906"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249685906"}]},"ts":"1676249685906"} 2023-02-13 00:54:45,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure fb3cb2309d059a71828c00de44bd7e7c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:46,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:46,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:46,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fb3cb2309d059a71828c00de44bd7e7c, disabling compactions & flushes 2023-02-13 00:54:46,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:46,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:46,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. after waiting 0 ms 2023-02-13 00:54:46,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:46,059 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-bottom] to archive 2023-02-13 00:54:46,061 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:46,067 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,069 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,070 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,072 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,105 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-bottom] to archive 2023-02-13 00:54:46,106 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:46,108 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,111 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,113 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,114 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:54:46,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:46,121 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c. 2023-02-13 00:54:46,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fb3cb2309d059a71828c00de44bd7e7c: Waiting for close lock at 1676249686054Running coprocessor pre-close hooks at 1676249686054Disabling compacts and flushes for region at 1676249686054Disabling writes for close at 1676249686054Writing region close event to WAL at 1676249686115 (+61 ms)Running coprocessor post-close hooks at 1676249686120 (+5 ms)Closed at 1676249686121 (+1 ms) 2023-02-13 00:54:46,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:46,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:46,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cd0b9cc8d658dd293604b3f54617114c, disabling compactions & flushes 2023-02-13 00:54:46,124 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:46,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:46,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. after waiting 0 ms 2023-02-13 00:54:46,124 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:46,124 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=fb3cb2309d059a71828c00de44bd7e7c, regionState=CLOSED 2023-02-13 00:54:46,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249686124"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249686124"}]},"ts":"1676249686124"} 2023-02-13 00:54:46,125 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6] to archive 2023-02-13 00:54:46,126 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:46,129 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/ac7fb191def14c40ad6263b54bbc4e59.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 00:54:46,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure fb3cb2309d059a71828c00de44bd7e7c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 219 msec 2023-02-13 00:54:46,131 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=fb3cb2309d059a71828c00de44bd7e7c, UNASSIGN in 235 msec 2023-02-13 00:54:46,133 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/132cf935ad694ac59f53608775ff97cb.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,135 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/6338d37bc23942b69325458d6326a1d7.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,136 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-409a1062d8dc4551b974cbbf35a3e205 2023-02-13 00:54:46,137 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-d64157c92491406bba607631996d473f 2023-02-13 00:54:46,139 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-c58ca86aa3d04a58bbf21f24a6545be6 2023-02-13 00:54:46,141 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1] to archive 2023-02-13 00:54:46,142 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:46,144 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/2c92faeb7bd740bb9664fbf0eae63a8d.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,146 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/5a6e9301366f47ca91ac0030d7c8c742.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,147 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/16bff9c2ffe24e348c04433ef8a8b67f.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,149 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/4845d941c0d04c698215c80981b45fbc.f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,150 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-5f42ebd1779c4cdca34e858e600f28c4 2023-02-13 00:54:46,151 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-b83ee500360943f999cb1befdaa653f9 2023-02-13 00:54:46,153 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/testSplitTableRegionUnevenDaughter=f8ce95eb513c26321825effddeecdacf-f91bf14383a249c1b46d658e882aaee1 2023-02-13 00:54:46,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:54:46,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:46,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c. 2023-02-13 00:54:46,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cd0b9cc8d658dd293604b3f54617114c: Waiting for close lock at 1676249686123Running coprocessor pre-close hooks at 1676249686123Disabling compacts and flushes for region at 1676249686123Disabling writes for close at 1676249686124 (+1 ms)Writing region close event to WAL at 1676249686154 (+30 ms)Running coprocessor post-close hooks at 1676249686159 (+5 ms)Closed at 1676249686160 (+1 ms) 2023-02-13 00:54:46,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:46,163 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=cd0b9cc8d658dd293604b3f54617114c, regionState=CLOSED 2023-02-13 00:54:46,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676249686163"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249686163"}]},"ts":"1676249686163"} 2023-02-13 00:54:46,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 00:54:46,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure cd0b9cc8d658dd293604b3f54617114c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 264 msec 2023-02-13 00:54:46,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-13 00:54:46,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=cd0b9cc8d658dd293604b3f54617114c, UNASSIGN in 273 msec 2023-02-13 00:54:46,171 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249686171"}]},"ts":"1676249686171"} 2023-02-13 00:54:46,173 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 00:54:46,175 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 00:54:46,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 296 msec 2023-02-13 00:54:46,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 00:54:46,202 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 00:54:46,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,215 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,217 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 00:54:46,226 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,226 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:46,226 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:46,230 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/recovered.edits] 2023-02-13 00:54:46,230 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/recovered.edits] 2023-02-13 00:54:46,230 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/recovered.edits] 2023-02-13 00:54:46,235 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/132cf935ad694ac59f53608775ff97cb 2023-02-13 00:54:46,235 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ea95b296150345d8af3b5eb00dfa3f0b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf1/ea95b296150345d8af3b5eb00dfa3f0b 2023-02-13 00:54:46,235 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/d746d17c2a674a0997eb324fbd08fbd4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf1/d746d17c2a674a0997eb324fbd08fbd4 2023-02-13 00:54:46,237 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/409a1062d8dc4551b974cbbf35a3e205 2023-02-13 00:54:46,239 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/e447649f67cf48e8a0bdbfc127f84fe4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/cf2/e447649f67cf48e8a0bdbfc127f84fe4 2023-02-13 00:54:46,240 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/6338d37bc23942b69325458d6326a1d7 2023-02-13 00:54:46,240 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/19c48c975dab490ab1b6df27f923fa15 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/cf2/19c48c975dab490ab1b6df27f923fa15 2023-02-13 00:54:46,243 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/a6c0db150b4c4c09bf7bf075f9a6fcec 2023-02-13 00:54:46,245 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c/recovered.edits/90.seqid 2023-02-13 00:54:46,245 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/ac7fb191def14c40ad6263b54bbc4e59 2023-02-13 00:54:46,246 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/fb3cb2309d059a71828c00de44bd7e7c 2023-02-13 00:54:46,247 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c/recovered.edits/90.seqid 2023-02-13 00:54:46,248 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:46,248 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/c58ca86aa3d04a58bbf21f24a6545be6 2023-02-13 00:54:46,249 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf1/d64157c92491406bba607631996d473f 2023-02-13 00:54:46,252 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/16bff9c2ffe24e348c04433ef8a8b67f 2023-02-13 00:54:46,254 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/2c92faeb7bd740bb9664fbf0eae63a8d 2023-02-13 00:54:46,255 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/4845d941c0d04c698215c80981b45fbc 2023-02-13 00:54:46,256 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5a6e9301366f47ca91ac0030d7c8c742 2023-02-13 00:54:46,258 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/5f42ebd1779c4cdca34e858e600f28c4 2023-02-13 00:54:46,259 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/b83ee500360943f999cb1befdaa653f9 2023-02-13 00:54:46,261 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/cf2/f91bf14383a249c1b46d658e882aaee1 2023-02-13 00:54:46,265 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf/recovered.edits/85.seqid 2023-02-13 00:54:46,266 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionUnevenDaughter/f8ce95eb513c26321825effddeecdacf 2023-02-13 00:54:46,266 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 00:54:46,269 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,291 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 00:54:46,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 00:54:46,333 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 00:54:46,334 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,334 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 00:54:46,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249686334"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:46,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249686334"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:46,335 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249686334"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:46,337 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:54:46,338 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f8ce95eb513c26321825effddeecdacf, NAME => 'testSplitTableRegionUnevenDaughter,,1676249681178.f8ce95eb513c26321825effddeecdacf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => fb3cb2309d059a71828c00de44bd7e7c, NAME => 'testSplitTableRegionUnevenDaughter,,1676249683926.fb3cb2309d059a71828c00de44bd7e7c.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => cd0b9cc8d658dd293604b3f54617114c, NAME => 'testSplitTableRegionUnevenDaughter,26,1676249683926.cd0b9cc8d658dd293604b3f54617114c.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 00:54:46,338 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 00:54:46,338 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249686338"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:46,340 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 00:54:46,342 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:46,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 135 msec 2023-02-13 00:54:46,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 00:54:46,523 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 00:54:46,539 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 358) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39230 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:56870 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39256 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34168 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39240 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34166 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:50522 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34158 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:33940 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:41539-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:59194 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:46226 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:59080 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:56874 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:38972 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39218 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34164 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=856 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 216), ProcessCount=168 (was 168), AvailableMemoryMB=3686 (was 3832) 2023-02-13 00:54:46,554 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=856, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=168, AvailableMemoryMB=3686 2023-02-13 00:54:46,554 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:46,561 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:54:46,561 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:54:46,563 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:46,564 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:46,565 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:46,567 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,568 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 empty. 2023-02-13 00:54:46,568 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,568 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 00:54:46,580 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:46,581 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7978078ab37756a938cfcf56fec698c9, NAME => 'testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 7978078ab37756a938cfcf56fec698c9, disabling compactions & flushes 2023-02-13 00:54:46,591 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. after waiting 0 ms 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,591 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,591 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 7978078ab37756a938cfcf56fec698c9: Waiting for close lock at 1676249686591Disabling compacts and flushes for region at 1676249686591Disabling writes for close at 1676249686591Writing region close event to WAL at 1676249686591Closed at 1676249686591 2023-02-13 00:54:46,594 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:46,595 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249686594"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249686594"}]},"ts":"1676249686594"} 2023-02-13 00:54:46,597 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:46,598 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:46,598 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249686598"}]},"ts":"1676249686598"} 2023-02-13 00:54:46,599 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 00:54:46,602 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:54:46,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:54:46,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:54:46,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:54:46,603 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:54:46,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, ASSIGN}] 2023-02-13 00:54:46,605 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, ASSIGN 2023-02-13 00:54:46,606 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:46,756 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:54:46,758 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:46,758 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249686758"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249686758"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249686758"}]},"ts":"1676249686758"} 2023-02-13 00:54:46,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:46,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7978078ab37756a938cfcf56fec698c9, NAME => 'testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:46,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,918 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,919 DEBUG [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1 2023-02-13 00:54:46,919 DEBUG [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1 2023-02-13 00:54:46,920 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7978078ab37756a938cfcf56fec698c9 columnFamilyName cf1 2023-02-13 00:54:46,920 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] regionserver.HStore(310): Store=7978078ab37756a938cfcf56fec698c9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:46,921 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,922 DEBUG [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2 2023-02-13 00:54:46,922 DEBUG [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2 2023-02-13 00:54:46,923 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7978078ab37756a938cfcf56fec698c9 columnFamilyName cf2 2023-02-13 00:54:46,924 INFO [StoreOpener-7978078ab37756a938cfcf56fec698c9-1] regionserver.HStore(310): Store=7978078ab37756a938cfcf56fec698c9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:46,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:46,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:46,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:46,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7978078ab37756a938cfcf56fec698c9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69878729, jitterRate=0.04127420485019684}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:46,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7978078ab37756a938cfcf56fec698c9: Running coprocessor pre-open hook at 1676249686916Writing region info on filesystem at 1676249686916Initializing all the Stores at 1676249686917 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249686917Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249686918 (+1 ms)Cleaning up temporary data from old regions at 1676249686926 (+8 ms)Cleaning up detritus from prior splits at 1676249686926Running coprocessor post-open hooks at 1676249686931 (+5 ms)Region opened successfully at 1676249686931 2023-02-13 00:54:46,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9., pid=27, masterSystemTime=1676249686912 2023-02-13 00:54:46,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:46,935 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:46,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249686935"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249686935"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249686935"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249686935"}]},"ts":"1676249686935"} 2023-02-13 00:54:46,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 00:54:46,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 in 177 msec 2023-02-13 00:54:46,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 00:54:46,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, ASSIGN in 336 msec 2023-02-13 00:54:46,943 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:46,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249686943"}]},"ts":"1676249686943"} 2023-02-13 00:54:46,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 00:54:46,947 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:46,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 386 msec 2023-02-13 00:54:47,074 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:47,076 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:47,076 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,077 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:47,077 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:47,077 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,077 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,079 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,079 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,079 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,079 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,081 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,081 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,081 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,082 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,082 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,082 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,084 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,085 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,085 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,085 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:54:47,085 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,086 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,097 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/db94761affd14843b745d0070e5d0313 2023-02-13 00:54:47,111 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a 2023-02-13 00:54:47,116 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/db94761affd14843b745d0070e5d0313 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 2023-02-13 00:54:47,121 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:47,122 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a 2023-02-13 00:54:47,127 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:47,128 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 43ms, sequenceid=6, compaction requested=false 2023-02-13 00:54:47,128 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 00:54:47,128 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:47,128 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,129 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,129 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,129 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,129 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,131 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:47,133 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72f66a65[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:47,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,134 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,135 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:47,136 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,136 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,137 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,138 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:47,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,142 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:47,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:47,142 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,146 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,146 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,147 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,148 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,149 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,149 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,152 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:47,153 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,152 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,153 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,154 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,254 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:47,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:47,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:47,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:47,275 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:47,275 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,276 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:47,276 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:47,276 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,276 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,278 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,278 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,278 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,278 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,279 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,280 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,281 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,283 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,283 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,283 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,283 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:47,283 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:47,285 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,287 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,297 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/ab0bf6f995c443aca2907a990f3e8429 2023-02-13 00:54:47,312 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/6ffa4bf350b54da8b7188121bf985e76 2023-02-13 00:54:47,317 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/ab0bf6f995c443aca2907a990f3e8429 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 2023-02-13 00:54:47,322 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:47,323 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/6ffa4bf350b54da8b7188121bf985e76 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 2023-02-13 00:54:47,329 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:47,330 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 47ms, sequenceid=19, compaction requested=false 2023-02-13 00:54:47,330 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:47,330 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,332 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@730067c6[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,334 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:47,334 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,336 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,342 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,344 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,344 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,344 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:47,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,348 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:47,348 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,348 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,348 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:47,348 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,350 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,351 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,351 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,354 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,354 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,354 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,357 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,357 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,358 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,359 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,358 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:47,359 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,358 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,359 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:47,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,360 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,405 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 00:54:47,405 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 00:54:47,406 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:47,406 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 00:54:47,460 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:47,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:47,482 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:47,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:47,484 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:47,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,485 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:47,485 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:47,486 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,486 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,490 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,491 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,490 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,492 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,492 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,492 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,492 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,492 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:47,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,494 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,495 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,496 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,496 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,520 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/80c08f13aaa54601a1efe19bd0c327d7 2023-02-13 00:54:47,536 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/31a3840d8f044bd7a8a6dcb3483a1610 2023-02-13 00:54:47,541 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/80c08f13aaa54601a1efe19bd0c327d7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 2023-02-13 00:54:47,547 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:47,548 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/31a3840d8f044bd7a8a6dcb3483a1610 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 2023-02-13 00:54:47,552 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:47,553 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 61ms, sequenceid=32, compaction requested=true 2023-02-13 00:54:47,553 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:47,554 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,555 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,555 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,556 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,556 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:47,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f2b0dc8[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,558 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:47,558 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,560 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,560 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,560 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:47,561 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,561 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,565 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,565 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,566 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,567 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,566 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,567 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:47,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,570 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:47,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:47,570 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,573 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,574 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,574 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,575 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,576 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,576 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,581 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:47,581 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,581 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,581 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:47,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,582 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitWithoutPONR because node does not exist (not an error) 2023-02-13 00:54:47,682 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:47,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:47,702 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:47,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:47,704 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:47,704 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,704 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:47,704 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:47,705 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,705 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,706 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,706 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,706 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,706 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,707 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,708 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,708 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,708 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,708 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,710 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,710 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,710 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,710 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,711 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,711 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,711 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,710 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,711 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,713 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,711 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,713 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,713 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,713 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,713 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,713 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,713 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:47,713 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:47,716 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,716 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,717 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,717 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,717 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,717 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,723 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/bef2ca6b0201452c87d12bb941395439 2023-02-13 00:54:47,737 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/190066c0c23b4ce9afda8e65c9dccd49 2023-02-13 00:54:47,742 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/bef2ca6b0201452c87d12bb941395439 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 2023-02-13 00:54:47,748 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:47,749 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/190066c0c23b4ce9afda8e65c9dccd49 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 2023-02-13 00:54:47,754 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:47,755 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 42ms, sequenceid=45, compaction requested=true 2023-02-13 00:54:47,755 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:47,755 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,755 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,755 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,755 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,755 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,755 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,758 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,758 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,758 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,758 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:47,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71b52668[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,760 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:47,760 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,761 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,762 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,762 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:47,762 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:47,763 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,763 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,763 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,765 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,764 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:47,770 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,770 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:47,770 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,770 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,770 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:47,770 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,773 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,773 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,774 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,776 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,776 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,776 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:47,780 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,780 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:47,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,781 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,881 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:47,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:47,901 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:47,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:47,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:47,903 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:47,903 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,903 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:47,903 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:47,904 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,904 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,905 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,905 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,906 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:47,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:47,907 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,907 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,907 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,907 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,907 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:47,908 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:47,908 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,908 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:47,908 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:47,909 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,911 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,911 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,911 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,911 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,911 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,922 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/b7cdec15066a498c91c9639989e7847f 2023-02-13 00:54:47,936 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/161accda38494cc7bca8f18c48a86077 2023-02-13 00:54:47,942 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/b7cdec15066a498c91c9639989e7847f as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f 2023-02-13 00:54:47,946 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:47,947 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/161accda38494cc7bca8f18c48a86077 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 2023-02-13 00:54:47,951 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:47,953 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 45ms, sequenceid=58, compaction requested=true 2023-02-13 00:54:47,953 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:47,953 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:47,953 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:47,953 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:47,953 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,953 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:47,953 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,955 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,955 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:47,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:47,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30d1df79[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:47,957 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:47,957 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,959 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,959 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:47,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:47,961 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,961 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,961 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,963 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,963 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:47,963 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,963 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,963 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,964 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:47,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:47,967 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:47,967 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:47,967 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:47,967 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:47,967 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,968 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:47,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,969 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,969 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:47,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:47,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,971 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,971 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:47,972 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:47,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,980 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:47,980 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,980 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:47,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:47,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:47,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:47,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:47,981 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:47,982 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,081 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:48,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:48,101 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:48,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:48,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 00:54:48,103 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 00:54:48,103 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,103 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 00:54:48,103 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 00:54:48,103 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,103 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,105 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,105 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,105 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,105 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,106 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,106 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 00:54:48,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:48,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:48,107 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:48,107 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:48,108 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:48,108 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:48,108 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. started... 2023-02-13 00:54:48,108 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:48,109 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,110 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,110 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,110 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:48,110 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,110 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:48,121 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 2023-02-13 00:54:48,137 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/c37ae064562446aabb89f3da5004c684 2023-02-13 00:54:48,142 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 2023-02-13 00:54:48,147 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:48,148 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/c37ae064562446aabb89f3da5004c684 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 2023-02-13 00:54:48,153 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:48,154 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 46ms, sequenceid=71, compaction requested=true 2023-02-13 00:54:48,155 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7978078ab37756a938cfcf56fec698c9: 2023-02-13 00:54:48,155 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:48,155 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:48,155 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:48,155 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,155 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 00:54:48,155 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 00:54:48,156 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,156 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:48,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 00:54:48,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@409fb0f2[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:48,158 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 00:54:48,159 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:48,161 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:48,162 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:48,162 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,162 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:48,163 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 00:54:48,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:48,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 00:54:48,166 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 00:54:48,166 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:48,166 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:48,166 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 00:54:48,166 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,168 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:48,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,171 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:48,171 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 00:54:48,171 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 00:54:48,176 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:48,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,176 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,176 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:48,176 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:48,175 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:48,176 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,176 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:48,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 00:54:48,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 00:54:48,276 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:48,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:48,540 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 00:54:48,541 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:48,542 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,543 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,543 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,544 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,545 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:48,545 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:48,545 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:48,793 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,794 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:48,794 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:48,794 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:48,796 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 00:54:48,797 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:48,797 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:48,797 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:48,797 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:48,798 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:48,801 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 00:54:48,803 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:48,803 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:48,804 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:48,804 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:48,804 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:48,805 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:48,805 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:48,806 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:48,806 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:48,806 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,806 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:48,807 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,807 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,807 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,807 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 00:54:48,808 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:48,808 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:48,809 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249718809 2023-02-13 00:54:48,810 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:48,810 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,814 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:48,814 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:48,815 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:48,815 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249748815 2023-02-13 00:54:48,815 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249808815 2023-02-13 00:54:48,815 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:48,815 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:48,817 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:48,817 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:48,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:48,817 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:49,065 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,065 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:49,065 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:49,065 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:49,065 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:49,066 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:49,066 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:49,066 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:49,066 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:49,067 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:49,068 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 00:54:49,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:49,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:49,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:49,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:49,070 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,070 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:49,070 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,070 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,071 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:49,071 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,071 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,071 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,071 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,072 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,072 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:49,072 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:49,073 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:49,073 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249719073 2023-02-13 00:54:49,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,075 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:49,077 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,077 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,078 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:49,078 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249749078 2023-02-13 00:54:49,078 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249809078 2023-02-13 00:54:49,078 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:49,078 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:49,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN}] 2023-02-13 00:54:49,080 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:49,080 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:49,080 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:49,328 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,328 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:49,328 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:49,328 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:49,328 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:49,329 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:49,329 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:49,329 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:49,329 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:49,329 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:49,331 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 00:54:49,331 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:49,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:49,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:49,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:49,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:49,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:49,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,334 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,334 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,334 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,334 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,334 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:49,335 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:49,335 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:49,336 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249719336 2023-02-13 00:54:49,336 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:49,337 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,340 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,340 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,340 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:49,340 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249749340 2023-02-13 00:54:49,341 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249809341 2023-02-13 00:54:49,341 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:49,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN}] 2023-02-13 00:54:49,341 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:49,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,344 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,344 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249689344"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249689344"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249689344"}]},"ts":"1676249689344"} 2023-02-13 00:54:49,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:49,346 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:49,346 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,346 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:49,346 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:49,591 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:49,591 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:49,592 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:49,592 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:49,592 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:49,594 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:49,594 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:49,594 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:49,595 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:49,595 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:49,595 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:49,596 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,596 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:49,597 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,597 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,597 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:49,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,599 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,599 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,599 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,599 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,599 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,600 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,600 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:54:49,600 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:49,601 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:49,601 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9 to restore RIT 2023-02-13 00:54:49,601 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249719601 2023-02-13 00:54:49,602 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:49,602 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,605 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,605 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,605 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:49,605 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249749605 2023-02-13 00:54:49,605 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249809605 2023-02-13 00:54:49,605 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:49,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249689605"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249689605"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249689605"}]},"ts":"1676249689605"} 2023-02-13 00:54:49,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:49,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:49,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:49,760 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7978078ab37756a938cfcf56fec698c9, disabling compactions & flushes 2023-02-13 00:54:49,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:49,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:49,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. after waiting 0 ms 2023-02-13 00:54:49,761 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:49,761 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7978078ab37756a938cfcf56fec698c9 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:54:49,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/68c25556fa3e42ffba2e00112e3be3bf 2023-02-13 00:54:49,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/bb618bced6134b49914aee3ee253d364 2023-02-13 00:54:49,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf1/68c25556fa3e42ffba2e00112e3be3bf as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf 2023-02-13 00:54:49,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:49,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/.tmp/cf2/bb618bced6134b49914aee3ee253d364 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 2023-02-13 00:54:49,800 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:49,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 40ms, sequenceid=82, compaction requested=true 2023-02-13 00:54:49,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:54:49,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:49,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. 2023-02-13 00:54:49,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7978078ab37756a938cfcf56fec698c9: Waiting for close lock at 1676249689760Running coprocessor pre-close hooks at 1676249689760Disabling compacts and flushes for region at 1676249689760Disabling writes for close at 1676249689761 (+1 ms)Obtaining lock to block concurrent updates at 1676249689761Preparing flush snapshotting stores in 7978078ab37756a938cfcf56fec698c9 at 1676249689761Finished memstore snapshotting testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249689761Flushing stores of testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. at 1676249689761Flushing 7978078ab37756a938cfcf56fec698c9/cf1: creating writer at 1676249689762 (+1 ms)Flushing 7978078ab37756a938cfcf56fec698c9/cf1: appending metadata at 1676249689765 (+3 ms)Flushing 7978078ab37756a938cfcf56fec698c9/cf1: closing flushed file at 1676249689765Flushing 7978078ab37756a938cfcf56fec698c9/cf2: creating writer at 1676249689776 (+11 ms)Flushing 7978078ab37756a938cfcf56fec698c9/cf2: appending metadata at 1676249689778 (+2 ms)Flushing 7978078ab37756a938cfcf56fec698c9/cf2: closing flushed file at 1676249689778Flushing 7978078ab37756a938cfcf56fec698c9/cf1: reopening flushed file at 1676249689791 (+13 ms)Flushing 7978078ab37756a938cfcf56fec698c9/cf2: reopening flushed file at 1676249689796 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7978078ab37756a938cfcf56fec698c9 in 40ms, sequenceid=82, compaction requested=true at 1676249689801 (+5 ms)Writing region close event to WAL at 1676249689818 (+17 ms)Running coprocessor post-close hooks at 1676249689823 (+5 ms)Closed at 1676249689823 2023-02-13 00:54:49,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:49,826 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED 2023-02-13 00:54:49,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249689826"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249689826"}]},"ts":"1676249689826"} 2023-02-13 00:54:49,827 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:49,827 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:49,827 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:49,855 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,856 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:49,856 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:49,856 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:49,856 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:49,857 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:49,857 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:49,857 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:49,857 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:49,858 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:49,859 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:49,859 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 00:54:49,859 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 00:54:49,860 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:49,860 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:49,861 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:49,861 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:49,861 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,862 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:49,862 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,863 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:49,863 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:49,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,865 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,865 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,865 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,865 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:49,865 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,865 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,865 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:49,866 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:54:49,866 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:49,867 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:49,867 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9 to restore RIT 2023-02-13 00:54:49,867 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249719867 2023-02-13 00:54:49,868 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:49,868 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,870 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,870 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:49,871 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:49,871 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249749871 2023-02-13 00:54:49,871 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249809871 2023-02-13 00:54:49,871 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:49,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:49,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249689871"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249689871"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249689871"}]},"ts":"1676249689871"} 2023-02-13 00:54:49,873 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:49,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 00:54:49,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 in 266 msec 2023-02-13 00:54:49,876 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:49,876 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:49,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:49,876 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:50,121 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,121 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:50,121 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:50,121 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:50,122 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:50,123 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:50,123 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:50,123 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:50,123 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:50,124 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:50,125 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:50,125 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 00:54:50,125 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:50,125 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:50,126 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:50,126 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:50,126 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:50,126 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,127 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:50,127 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,127 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,127 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:50,128 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,128 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:50,128 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:50,128 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:50,128 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,128 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,128 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,129 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,129 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,129 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:50,129 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:50,130 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:50,130 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9 to restore RIT 2023-02-13 00:54:50,130 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249720130 2023-02-13 00:54:50,131 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:50,131 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:50,133 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,134 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,134 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:50,134 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249750134 2023-02-13 00:54:50,134 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249810134 2023-02-13 00:54:50,134 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:50,134 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:50,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:50,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 00:54:50,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN in 794 msec 2023-02-13 00:54:50,141 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:50,141 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:50,141 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:50,384 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,384 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:50,384 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:50,384 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:50,385 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:50,385 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:50,385 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:50,385 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:50,385 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:50,386 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:50,387 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:50,387 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:50,387 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:50,387 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:50,387 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:50,388 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:50,388 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:50,388 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,389 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:50,389 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,389 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,389 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,390 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,390 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:50,391 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:50,391 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:50,392 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249720392 2023-02-13 00:54:50,392 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:50,393 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,395 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,396 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,396 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:50,396 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249750396 2023-02-13 00:54:50,396 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249810396 2023-02-13 00:54:50,396 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:50,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,396 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:50,411 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7978078ab37756a938cfcf56fec698c9, threads=14 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,412 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,413 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,548 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,555 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,560 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,568 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,578 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,580 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,581 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,581 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,582 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,583 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,584 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,584 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,585 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,585 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,585 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7978078ab37756a938cfcf56fec698c9 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:54:50,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:50,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:50,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:50,646 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,646 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:50,646 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:50,646 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:50,647 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:50,647 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:50,647 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:50,647 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:50,647 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:50,648 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:50,648 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 00:54:50,649 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:50,649 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:50,649 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:50,650 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:50,650 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:50,650 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:50,651 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,651 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:50,651 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,652 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:50,652 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:50,653 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,653 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:50,653 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:50,654 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:50,654 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,654 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,654 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,654 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,654 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:50,654 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 00:54:50,655 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:50,655 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:50,656 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249720656 2023-02-13 00:54:50,657 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:50,657 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,660 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,660 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:50,660 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:50,660 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249750660 2023-02-13 00:54:50,660 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249810660 2023-02-13 00:54:50,660 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:50,670 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7978078ab37756a938cfcf56fec698c9, threads=14 2023-02-13 00:54:50,671 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,671 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,671 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,672 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,672 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,672 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,674 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,675 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,675 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,675 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,675 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,676 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,680 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,680 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,795 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,823 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:50,824 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,824 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,832 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,832 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,840 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,862 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,872 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,883 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,892 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,895 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,905 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,906 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,909 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 for region: 7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:50,909 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7978078ab37756a938cfcf56fec698c9 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:54:50,965 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:50,971 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:50,973 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:50,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:50,973 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:50,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:50,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:51,161 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,161 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:51,161 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:51,161 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:51,162 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:51,162 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:51,162 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:51,162 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:51,162 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:51,163 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:51,164 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 00:54:51,164 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:51,164 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:51,165 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:51,165 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:51,165 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:51,165 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:51,166 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,166 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:51,166 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,166 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,167 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:51,168 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,168 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,168 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:51,168 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:51,168 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:51,169 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:51,170 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:51,170 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249721170 2023-02-13 00:54:51,171 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:51,171 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,174 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:51,174 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:51,175 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:51,175 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249751175 2023-02-13 00:54:51,175 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249811175 2023-02-13 00:54:51,175 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 00:54:51,178 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:51,180 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:51,180 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:51,180 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:51,425 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:51,425 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:51,425 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:51,425 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:51,426 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:51,427 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:51,427 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:51,427 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:51,427 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:51,427 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:51,428 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 00:54:51,429 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:51,429 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:51,429 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:51,430 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:51,430 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:51,430 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:51,430 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,431 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:51,431 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,431 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:51,432 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:51,432 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,433 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN 2023-02-13 00:54:51,433 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,433 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:51,433 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,433 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,433 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,433 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7978078ab37756a938cfcf56fec698c9, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:51,433 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7978078ab37756a938cfcf56fec698c9, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:51,434 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:51,434 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:51,435 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:51,435 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249721435 2023-02-13 00:54:51,436 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:51,436 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,439 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:51,439 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=7978078ab37756a938cfcf56fec698c9, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:51,439 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:51,439 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249751439 2023-02-13 00:54:51,439 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249811439 2023-02-13 00:54:51,439 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:51,441 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249691441"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676249691441"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676249691441"}]},"ts":"1676249691441"} 2023-02-13 00:54:51,442 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691441"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249691441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249691441"}]},"ts":"1676249691441"} 2023-02-13 00:54:51,442 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691441"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249691441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249691441"}]},"ts":"1676249691441"} 2023-02-13 00:54:51,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, ASSIGN}] 2023-02-13 00:54:51,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, ASSIGN 2023-02-13 00:54:51,449 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:51,452 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, ASSIGN 2023-02-13 00:54:51,453 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:51,599 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:54:51,601 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3dd96bf9a4733d352bef76a275164362, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,601 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691601"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249691601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249691601"}]},"ts":"1676249691601"} 2023-02-13 00:54:51,603 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 3dd96bf9a4733d352bef76a275164362, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:51,607 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=99cb1f102b60382a91299eb121580efb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,607 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691607"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249691607"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249691607"}]},"ts":"1676249691607"} 2023-02-13 00:54:51,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 99cb1f102b60382a91299eb121580efb, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:51,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 99cb1f102b60382a91299eb121580efb, NAME => 'testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 00:54:51,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:51,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:51,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,761 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,762 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1 2023-02-13 00:54:51,762 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1 2023-02-13 00:54:51,762 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99cb1f102b60382a91299eb121580efb columnFamilyName cf1 2023-02-13 00:54:51,778 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-top 2023-02-13 00:54:51,782 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-top 2023-02-13 00:54:51,787 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-top 2023-02-13 00:54:51,792 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-top 2023-02-13 00:54:51,797 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-top 2023-02-13 00:54:51,802 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-top 2023-02-13 00:54:51,806 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-top 2023-02-13 00:54:51,806 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(310): Store=99cb1f102b60382a91299eb121580efb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:51,807 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,807 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2 2023-02-13 00:54:51,807 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2 2023-02-13 00:54:51,808 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 99cb1f102b60382a91299eb121580efb columnFamilyName cf2 2023-02-13 00:54:51,823 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-top 2023-02-13 00:54:51,827 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-top 2023-02-13 00:54:51,832 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-top 2023-02-13 00:54:51,836 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-top 2023-02-13 00:54:51,840 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-top 2023-02-13 00:54:51,845 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-top 2023-02-13 00:54:51,852 DEBUG [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-top 2023-02-13 00:54:51,852 INFO [StoreOpener-99cb1f102b60382a91299eb121580efb-1] regionserver.HStore(310): Store=99cb1f102b60382a91299eb121580efb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:51,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,856 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:51,857 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:51,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 99cb1f102b60382a91299eb121580efb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64039665, jitterRate=-0.04573462903499603}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:51,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 99cb1f102b60382a91299eb121580efb: Running coprocessor pre-open hook at 1676249691760Writing region info on filesystem at 1676249691760Initializing all the Stores at 1676249691761 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249691761Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249691761Cleaning up temporary data from old regions at 1676249691855 (+94 ms)Cleaning up detritus from prior splits at 1676249691855Running coprocessor post-open hooks at 1676249691858 (+3 ms)Region opened successfully at 1676249691858 2023-02-13 00:54:51,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb., pid=34, masterSystemTime=1676249691755 2023-02-13 00:54:51,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:51,860 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:51,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:54:51,860 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:51,860 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,860 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,860 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): 99cb1f102b60382a91299eb121580efb/cf1 is initiating minor compaction (all files) 2023-02-13 00:54:51,860 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): 99cb1f102b60382a91299eb121580efb/cf2 is initiating minor compaction (all files) 2023-02-13 00:54:51,860 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 99cb1f102b60382a91299eb121580efb/cf1 in testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,860 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 99cb1f102b60382a91299eb121580efb/cf2 in testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,861 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/.tmp, totalSize=34.9 K 2023-02-13 00:54:51,861 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/.tmp, totalSize=34.9 K 2023-02-13 00:54:51,862 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249687067 2023-02-13 00:54:51,862 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249687067 2023-02-13 00:54:51,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:51,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3dd96bf9a4733d352bef76a275164362, NAME => 'testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 00:54:51,862 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249687256 2023-02-13 00:54:51,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:51,863 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249687256 2023-02-13 00:54:51,863 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=99cb1f102b60382a91299eb121580efb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:51,863 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691863"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249691863"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249691863"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249691863"}]},"ts":"1676249691863"} 2023-02-13 00:54:51,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,863 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249687463 2023-02-13 00:54:51,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,863 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249687463 2023-02-13 00:54:51,864 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249687685 2023-02-13 00:54:51,864 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249687685 2023-02-13 00:54:51,865 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249687884 2023-02-13 00:54:51,865 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249687884 2023-02-13 00:54:51,865 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249688084 2023-02-13 00:54:51,865 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,865 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249688084 2023-02-13 00:54:51,866 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249688279 2023-02-13 00:54:51,866 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249688279 2023-02-13 00:54:51,868 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1 2023-02-13 00:54:51,868 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1 2023-02-13 00:54:51,868 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3dd96bf9a4733d352bef76a275164362 columnFamilyName cf1 2023-02-13 00:54:51,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-02-13 00:54:51,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 99cb1f102b60382a91299eb121580efb, server=jenkins-hbase4.apache.org,41539,1676249678442 in 257 msec 2023-02-13 00:54:51,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, ASSIGN in 424 msec 2023-02-13 00:54:51,885 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): 99cb1f102b60382a91299eb121580efb#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:51,886 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): 99cb1f102b60382a91299eb121580efb#cf1#compaction#33 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:51,890 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-bottom 2023-02-13 00:54:51,898 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-bottom 2023-02-13 00:54:51,906 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-bottom 2023-02-13 00:54:51,915 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-bottom 2023-02-13 00:54:51,920 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-bottom 2023-02-13 00:54:51,921 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/.tmp/cf2/551fe323f65f40b884cb9777342cada9 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/551fe323f65f40b884cb9777342cada9 2023-02-13 00:54:51,926 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-bottom 2023-02-13 00:54:51,930 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 99cb1f102b60382a91299eb121580efb/cf2 of 99cb1f102b60382a91299eb121580efb into 551fe323f65f40b884cb9777342cada9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:51,930 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 99cb1f102b60382a91299eb121580efb: 2023-02-13 00:54:51,930 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb., storeName=99cb1f102b60382a91299eb121580efb/cf2, priority=9, startTime=1676249691860; duration=0sec 2023-02-13 00:54:51,930 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:51,932 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/.tmp/cf1/15c2e396d14f4858b9730415af02b510 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/15c2e396d14f4858b9730415af02b510 2023-02-13 00:54:51,932 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-bottom 2023-02-13 00:54:51,932 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(310): Store=3dd96bf9a4733d352bef76a275164362/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:51,933 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,934 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2 2023-02-13 00:54:51,934 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2 2023-02-13 00:54:51,934 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3dd96bf9a4733d352bef76a275164362 columnFamilyName cf2 2023-02-13 00:54:51,938 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 99cb1f102b60382a91299eb121580efb/cf1 of 99cb1f102b60382a91299eb121580efb into 15c2e396d14f4858b9730415af02b510(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:51,938 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 99cb1f102b60382a91299eb121580efb: 2023-02-13 00:54:51,938 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb., storeName=99cb1f102b60382a91299eb121580efb/cf1, priority=9, startTime=1676249691859; duration=0sec 2023-02-13 00:54:51,938 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:51,950 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-bottom 2023-02-13 00:54:51,954 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-bottom 2023-02-13 00:54:51,958 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-bottom 2023-02-13 00:54:51,963 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-bottom 2023-02-13 00:54:51,967 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-bottom 2023-02-13 00:54:51,972 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-bottom 2023-02-13 00:54:51,977 DEBUG [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-bottom 2023-02-13 00:54:51,977 INFO [StoreOpener-3dd96bf9a4733d352bef76a275164362-1] regionserver.HStore(310): Store=3dd96bf9a4733d352bef76a275164362/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:51,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:51,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:51,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3dd96bf9a4733d352bef76a275164362; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68178010, jitterRate=0.015931516885757446}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:51,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3dd96bf9a4733d352bef76a275164362: Running coprocessor pre-open hook at 1676249691863Writing region info on filesystem at 1676249691863Initializing all the Stores at 1676249691865 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249691865Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249691865Cleaning up temporary data from old regions at 1676249691980 (+115 ms)Cleaning up detritus from prior splits at 1676249691980Running coprocessor post-open hooks at 1676249691983 (+3 ms)Region opened successfully at 1676249691983 2023-02-13 00:54:51,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362., pid=33, masterSystemTime=1676249691755 2023-02-13 00:54:51,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:51,985 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:51,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:54:51,985 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,985 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): 3dd96bf9a4733d352bef76a275164362/cf1 is initiating minor compaction (all files) 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,985 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): 3dd96bf9a4733d352bef76a275164362/cf2 is initiating minor compaction (all files) 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 3dd96bf9a4733d352bef76a275164362/cf1 in testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 3dd96bf9a4733d352bef76a275164362/cf2 in testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/.tmp, totalSize=34.9 K 2023-02-13 00:54:51,985 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/.tmp, totalSize=34.9 K 2023-02-13 00:54:51,986 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249687067 2023-02-13 00:54:51,986 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249687067 2023-02-13 00:54:51,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:51,987 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249687256 2023-02-13 00:54:51,987 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249687256 2023-02-13 00:54:51,987 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249687463 2023-02-13 00:54:51,987 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249687463 2023-02-13 00:54:51,987 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=3dd96bf9a4733d352bef76a275164362, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:51,988 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249691987"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249691987"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249691987"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249691987"}]},"ts":"1676249691987"} 2023-02-13 00:54:51,988 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249687685 2023-02-13 00:54:51,988 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249687685 2023-02-13 00:54:51,988 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249687884 2023-02-13 00:54:51,988 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249687884 2023-02-13 00:54:51,989 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249688084 2023-02-13 00:54:51,989 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249688084 2023-02-13 00:54:51,989 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249688279 2023-02-13 00:54:51,990 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249688279 2023-02-13 00:54:51,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-02-13 00:54:51,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 3dd96bf9a4733d352bef76a275164362, server=jenkins-hbase4.apache.org,41539,1676249678442 in 386 msec 2023-02-13 00:54:51,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 00:54:51,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, ASSIGN in 547 msec 2023-02-13 00:54:51,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb in 3.4540 sec 2023-02-13 00:54:52,005 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): 3dd96bf9a4733d352bef76a275164362#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:52,006 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3dd96bf9a4733d352bef76a275164362#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:52,023 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/.tmp/cf2/ab238fb10ac74944a2e913094213c450 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/ab238fb10ac74944a2e913094213c450 2023-02-13 00:54:52,025 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/.tmp/cf1/6b7671a897c04a0c9cf01a6db36f473c as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/6b7671a897c04a0c9cf01a6db36f473c 2023-02-13 00:54:52,030 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3dd96bf9a4733d352bef76a275164362/cf2 of 3dd96bf9a4733d352bef76a275164362 into ab238fb10ac74944a2e913094213c450(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:52,030 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3dd96bf9a4733d352bef76a275164362: 2023-02-13 00:54:52,031 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362., storeName=3dd96bf9a4733d352bef76a275164362/cf2, priority=9, startTime=1676249691985; duration=0sec 2023-02-13 00:54:52,031 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:52,032 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3dd96bf9a4733d352bef76a275164362/cf1 of 3dd96bf9a4733d352bef76a275164362 into 6b7671a897c04a0c9cf01a6db36f473c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:52,032 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 3dd96bf9a4733d352bef76a275164362: 2023-02-13 00:54:52,032 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362., storeName=3dd96bf9a4733d352bef76a275164362/cf1, priority=9, startTime=1676249691984; duration=0sec 2023-02-13 00:54:52,032 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:52,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41539] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:40408 deadline: 1676249752190, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9. is not online on jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:52,303 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:54:52,311 INFO [Listener at localhost/43763] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 00:54:52,313 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:54:52,330 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:52,332 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 00:54:52,332 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-13 00:54:52,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:52,337 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249692337"}]},"ts":"1676249692337"} 2023-02-13 00:54:52,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 00:54:52,341 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 00:54:52,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, UNASSIGN}] 2023-02-13 00:54:52,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, UNASSIGN 2023-02-13 00:54:52,344 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=99cb1f102b60382a91299eb121580efb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:52,344 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249692343"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249692343"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249692343"}]},"ts":"1676249692343"} 2023-02-13 00:54:52,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 99cb1f102b60382a91299eb121580efb, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:52,348 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, UNASSIGN 2023-02-13 00:54:52,349 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3dd96bf9a4733d352bef76a275164362, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:52,349 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249692349"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249692349"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249692349"}]},"ts":"1676249692349"} 2023-02-13 00:54:52,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 3dd96bf9a4733d352bef76a275164362, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:52,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:52,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:52,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3dd96bf9a4733d352bef76a275164362, disabling compactions & flushes 2023-02-13 00:54:52,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:52,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:52,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. after waiting 0 ms 2023-02-13 00:54:52,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:52,509 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-bottom] to archive 2023-02-13 00:54:52,510 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:52,512 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,513 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,515 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,516 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,518 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,519 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,520 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,522 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-bottom] to archive 2023-02-13 00:54:52,523 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:52,524 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,526 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,527 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,528 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,529 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,530 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,531 DEBUG [StoreCloser-testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:54:52,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:52,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362. 2023-02-13 00:54:52,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3dd96bf9a4733d352bef76a275164362: Waiting for close lock at 1676249692498Running coprocessor pre-close hooks at 1676249692498Disabling compacts and flushes for region at 1676249692498Disabling writes for close at 1676249692498Writing region close event to WAL at 1676249692532 (+34 ms)Running coprocessor post-close hooks at 1676249692535 (+3 ms)Closed at 1676249692535 2023-02-13 00:54:52,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:52,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:52,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 99cb1f102b60382a91299eb121580efb, disabling compactions & flushes 2023-02-13 00:54:52,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:52,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:52,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. after waiting 0 ms 2023-02-13 00:54:52,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:52,537 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=3dd96bf9a4733d352bef76a275164362, regionState=CLOSED 2023-02-13 00:54:52,537 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249692537"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249692537"}]},"ts":"1676249692537"} 2023-02-13 00:54:52,538 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf-top] to archive 2023-02-13 00:54:52,539 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:52,541 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 00:54:52,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 3dd96bf9a4733d352bef76a275164362, server=jenkins-hbase4.apache.org,41539,1676249678442 in 189 msec 2023-02-13 00:54:52,542 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,543 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=3dd96bf9a4733d352bef76a275164362, UNASSIGN in 200 msec 2023-02-13 00:54:52,544 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,545 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,546 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,547 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,549 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,550 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364-top] to archive 2023-02-13 00:54:52,551 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:54:52,552 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,553 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,554 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,555 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,556 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,557 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,558 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:54:52,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:52,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb. 2023-02-13 00:54:52,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 99cb1f102b60382a91299eb121580efb: Waiting for close lock at 1676249692537Running coprocessor pre-close hooks at 1676249692537Disabling compacts and flushes for region at 1676249692537Disabling writes for close at 1676249692537Writing region close event to WAL at 1676249692559 (+22 ms)Running coprocessor post-close hooks at 1676249692562 (+3 ms)Closed at 1676249692562 2023-02-13 00:54:52,564 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:52,564 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=99cb1f102b60382a91299eb121580efb, regionState=CLOSED 2023-02-13 00:54:52,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249692564"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249692564"}]},"ts":"1676249692564"} 2023-02-13 00:54:52,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 00:54:52,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 99cb1f102b60382a91299eb121580efb, server=jenkins-hbase4.apache.org,41539,1676249678442 in 221 msec 2023-02-13 00:54:52,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-02-13 00:54:52,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=99cb1f102b60382a91299eb121580efb, UNASSIGN in 226 msec 2023-02-13 00:54:52,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249692571"}]},"ts":"1676249692571"} 2023-02-13 00:54:52,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 00:54:52,579 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 00:54:52,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 248 msec 2023-02-13 00:54:52,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 00:54:52,639 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 00:54:52,640 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-13 00:54:52,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,643 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 00:54:52,651 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,651 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:52,651 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:52,653 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/recovered.edits] 2023-02-13 00:54:52,654 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/recovered.edits] 2023-02-13 00:54:52,654 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/recovered.edits] 2023-02-13 00:54:52,657 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/.regioninfo to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/.regioninfo 2023-02-13 00:54:52,657 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/.regioninfo to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/.regioninfo 2023-02-13 00:54:52,658 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/1a25ee4317624fb99e94c3d78cdd9ca0 2023-02-13 00:54:52,660 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/68c25556fa3e42ffba2e00112e3be3bf 2023-02-13 00:54:52,661 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,661 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/1a25ee4317624fb99e94c3d78cdd9ca0.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,661 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/80c08f13aaa54601a1efe19bd0c327d7 2023-02-13 00:54:52,662 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,662 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/68c25556fa3e42ffba2e00112e3be3bf.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,663 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/ab0bf6f995c443aca2907a990f3e8429 2023-02-13 00:54:52,664 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,664 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/80c08f13aaa54601a1efe19bd0c327d7.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,664 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/b7cdec15066a498c91c9639989e7847f 2023-02-13 00:54:52,665 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,666 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/ab0bf6f995c443aca2907a990f3e8429.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,666 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/bef2ca6b0201452c87d12bb941395439 2023-02-13 00:54:52,667 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,667 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/b7cdec15066a498c91c9639989e7847f.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,667 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf1/db94761affd14843b745d0070e5d0313 2023-02-13 00:54:52,669 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,669 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/bef2ca6b0201452c87d12bb941395439.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,671 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,671 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf1/db94761affd14843b745d0070e5d0313.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,672 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/161accda38494cc7bca8f18c48a86077 2023-02-13 00:54:52,673 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/190066c0c23b4ce9afda8e65c9dccd49 2023-02-13 00:54:52,675 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,675 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/31a3840d8f044bd7a8a6dcb3483a1610 2023-02-13 00:54:52,682 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/161accda38494cc7bca8f18c48a86077.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,683 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a 2023-02-13 00:54:52,683 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,684 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/190066c0c23b4ce9afda8e65c9dccd49.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,685 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/6ffa4bf350b54da8b7188121bf985e76 2023-02-13 00:54:52,685 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,686 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/31a3840d8f044bd7a8a6dcb3483a1610.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,687 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/bb618bced6134b49914aee3ee253d364 2023-02-13 00:54:52,688 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,689 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/651edfd9f2034ae1bc45b6e8fac9ac0a.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,689 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/cf2/c37ae064562446aabb89f3da5004c684 2023-02-13 00:54:52,690 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,691 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/6ffa4bf350b54da8b7188121bf985e76.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,692 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,693 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/bb618bced6134b49914aee3ee253d364.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,694 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9/recovered.edits/85.seqid 2023-02-13 00:54:52,695 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/3dd96bf9a4733d352bef76a275164362/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,695 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,695 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/99cb1f102b60382a91299eb121580efb/cf2/c37ae064562446aabb89f3da5004c684.7978078ab37756a938cfcf56fec698c9 2023-02-13 00:54:52,698 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/6b7671a897c04a0c9cf01a6db36f473c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf1/6b7671a897c04a0c9cf01a6db36f473c 2023-02-13 00:54:52,698 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/15c2e396d14f4858b9730415af02b510 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf1/15c2e396d14f4858b9730415af02b510 2023-02-13 00:54:52,700 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/ab238fb10ac74944a2e913094213c450 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/cf2/ab238fb10ac74944a2e913094213c450 2023-02-13 00:54:52,701 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/551fe323f65f40b884cb9777342cada9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/cf2/551fe323f65f40b884cb9777342cada9 2023-02-13 00:54:52,704 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362/recovered.edits/90.seqid 2023-02-13 00:54:52,704 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb/recovered.edits/90.seqid 2023-02-13 00:54:52,705 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/3dd96bf9a4733d352bef76a275164362 2023-02-13 00:54:52,705 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitWithoutPONR/99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:52,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 00:54:52,707 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,710 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 00:54:52,712 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 00:54:52,713 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,713 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 00:54:52,713 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249692713"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:52,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249692713"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:52,714 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249692713"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:52,716 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:54:52,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7978078ab37756a938cfcf56fec698c9, NAME => 'testSplitWithoutPONR,,1676249686561.7978078ab37756a938cfcf56fec698c9.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 3dd96bf9a4733d352bef76a275164362, NAME => 'testSplitWithoutPONR,,1676249688541.3dd96bf9a4733d352bef76a275164362.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 99cb1f102b60382a91299eb121580efb, NAME => 'testSplitWithoutPONR,41,1676249688541.99cb1f102b60382a91299eb121580efb.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 00:54:52,716 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 00:54:52,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249692716"}]},"ts":"9223372036854775807"} 2023-02-13 00:54:52,718 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 00:54:52,720 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:52,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 79 msec 2023-02-13 00:54:52,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 00:54:52,746 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 00:54:52,761 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=396 (was 391) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39230 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:56870 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57238 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5c02c0c4-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39240 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57744 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57664 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57668 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34158 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:34338 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:33940 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:59194 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57682 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:46226 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:40052 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:59080 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:38972 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=858 (was 856) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 215) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 168), AvailableMemoryMB=3607 (was 3686) 2023-02-13 00:54:52,776 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=167, AvailableMemoryMB=3607 2023-02-13 00:54:52,776 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:52,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:54:52,779 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:54:52,781 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:52,782 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:54:52,783 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:54:52,785 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:52,785 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c empty. 2023-02-13 00:54:52,786 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:52,786 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 00:54:52,799 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 00:54:52,800 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => ea3eaa9ea79a3bf34eba60089ea7ec8c, NAME => 'testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing ea3eaa9ea79a3bf34eba60089ea7ec8c, disabling compactions & flushes 2023-02-13 00:54:52,810 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. after waiting 0 ms 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:52,810 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:52,810 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: Waiting for close lock at 1676249692810Disabling compacts and flushes for region at 1676249692810Disabling writes for close at 1676249692810Writing region close event to WAL at 1676249692810Closed at 1676249692810 2023-02-13 00:54:52,813 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:54:52,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249692814"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249692814"}]},"ts":"1676249692814"} 2023-02-13 00:54:52,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:54:52,816 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:54:52,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249692816"}]},"ts":"1676249692816"} 2023-02-13 00:54:52,818 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 00:54:52,824 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:54:52,826 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:54:52,826 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:54:52,826 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:54:52,826 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:54:52,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN}] 2023-02-13 00:54:52,828 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:52,828 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:54:52,979 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:54:52,980 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:52,980 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249692980"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249692980"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249692980"}]},"ts":"1676249692980"} 2023-02-13 00:54:52,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:53,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ea3eaa9ea79a3bf34eba60089ea7ec8c, NAME => 'testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:53,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:53,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:53,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,139 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,140 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 2023-02-13 00:54:53,141 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 2023-02-13 00:54:53,141 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea3eaa9ea79a3bf34eba60089ea7ec8c columnFamilyName cf1 2023-02-13 00:54:53,141 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(310): Store=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:53,142 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,143 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 2023-02-13 00:54:53,143 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 2023-02-13 00:54:53,143 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea3eaa9ea79a3bf34eba60089ea7ec8c columnFamilyName cf2 2023-02-13 00:54:53,144 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(310): Store=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:53,144 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,147 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:53,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:53,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:54:53,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ea3eaa9ea79a3bf34eba60089ea7ec8c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62082963, jitterRate=-0.07489176094532013}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:53,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: Running coprocessor pre-open hook at 1676249693138Writing region info on filesystem at 1676249693138Initializing all the Stores at 1676249693139 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249693139Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249693139Cleaning up temporary data from old regions at 1676249693146 (+7 ms)Cleaning up detritus from prior splits at 1676249693146Running coprocessor post-open hooks at 1676249693150 (+4 ms)Region opened successfully at 1676249693150 2023-02-13 00:54:53,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c., pid=43, masterSystemTime=1676249693134 2023-02-13 00:54:53,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,153 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,153 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249693153"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249693153"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249693153"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249693153"}]},"ts":"1676249693153"} 2023-02-13 00:54:53,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 00:54:53,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 172 msec 2023-02-13 00:54:53,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 00:54:53,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN in 330 msec 2023-02-13 00:54:53,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:54:53,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249693159"}]},"ts":"1676249693159"} 2023-02-13 00:54:53,161 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 00:54:53,165 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:54:53,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 386 msec 2023-02-13 00:54:53,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:53,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:53,291 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:53,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:53,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,293 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,293 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,293 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,293 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,294 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,294 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,295 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,296 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:53,296 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:53,296 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,296 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:53,297 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:54:53,298 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,302 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,310 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/170ffcfbdec445d0813fcf7c348fd80c 2023-02-13 00:54:53,323 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/7e142745db734ace983d997d66aea3f5 2023-02-13 00:54:53,328 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/170ffcfbdec445d0813fcf7c348fd80c as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c 2023-02-13 00:54:53,332 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:53,332 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/7e142745db734ace983d997d66aea3f5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 2023-02-13 00:54:53,336 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:54:53,337 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 40ms, sequenceid=6, compaction requested=false 2023-02-13 00:54:53,337 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,338 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:53,338 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,340 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,340 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,340 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,340 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:53,343 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:53,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67fdb064[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:53,343 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,344 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:53,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:53,345 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:53,347 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,347 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,347 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,348 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,348 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,348 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,349 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,349 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,349 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:53,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:53,353 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:53,353 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:53,353 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:53,353 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:53,353 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,355 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,356 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,357 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,358 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,359 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,359 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,362 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,362 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:53,362 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,362 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,361 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,362 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,362 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:53,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,362 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:53,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,362 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,363 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,462 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:53,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:53,482 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:53,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:53,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:53,484 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:53,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:53,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:53,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,489 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,491 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,491 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,491 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,491 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,491 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,491 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,492 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,493 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,493 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,494 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,495 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,495 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,495 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,495 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,496 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,496 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:53,496 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:53,496 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,496 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:53,496 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:53,513 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/70e6217dd7ce4a95924e683694741f00 2023-02-13 00:54:53,535 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/8e11cbb1a8004c50ad90cb28e9961614 2023-02-13 00:54:53,541 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/70e6217dd7ce4a95924e683694741f00 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 2023-02-13 00:54:53,546 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:53,547 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/8e11cbb1a8004c50ad90cb28e9961614 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 2023-02-13 00:54:53,551 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:54:53,552 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 56ms, sequenceid=19, compaction requested=false 2023-02-13 00:54:53,552 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:53,552 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,554 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:53,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f235c95[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:53,557 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:53,557 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,559 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,559 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,559 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:53,560 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,560 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:53,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:53,560 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,564 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,564 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,564 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,565 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,567 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,567 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,567 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:53,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:53,570 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:53,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:53,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:53,570 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:53,570 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,572 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,573 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,574 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,575 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,576 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,576 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,579 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,579 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,579 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,580 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:53,580 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:53,579 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,581 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,581 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,680 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:53,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:53,696 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:53,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:53,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:53,698 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,698 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,698 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:53,698 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:53,699 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,699 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,700 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,700 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,700 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,700 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,701 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,701 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:53,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,702 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,703 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:53,703 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,703 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,703 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:53,707 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,708 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,708 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,708 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,708 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,708 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,713 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/5de7c3e4e9074f42affca8eb62a9bce1 2023-02-13 00:54:53,732 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/bb605aa7640b48c691be8526e9468ed5 2023-02-13 00:54:53,737 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/5de7c3e4e9074f42affca8eb62a9bce1 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 2023-02-13 00:54:53,741 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:53,742 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/bb605aa7640b48c691be8526e9468ed5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 2023-02-13 00:54:53,746 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:54:53,747 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 44ms, sequenceid=32, compaction requested=true 2023-02-13 00:54:53,747 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:53,747 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,747 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:53,747 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:53,747 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,748 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,749 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:53,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b7661f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:53,751 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:53,752 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:53,753 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:53,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,755 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,756 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,756 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:53,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:53,761 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:53,761 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:53,761 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:53,761 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:53,761 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:53,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,763 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,764 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,764 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,766 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,766 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,767 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,778 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,778 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,779 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:53,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,778 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,778 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,779 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,779 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,780 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,880 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:53,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:53,895 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:53,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:53,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:53,897 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,897 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,897 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:53,897 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:53,898 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,898 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,899 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,899 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,900 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:53,900 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,901 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,901 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:53,901 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,902 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:53,901 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:53,902 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,902 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,902 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:53,903 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,904 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,904 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,904 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,904 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,904 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,912 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/fba7d27e3e13454abf830a904102c1f4 2023-02-13 00:54:53,924 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/39375d12364d4a7e9501f574233ae0d7 2023-02-13 00:54:53,929 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/fba7d27e3e13454abf830a904102c1f4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 2023-02-13 00:54:53,934 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:53,934 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/39375d12364d4a7e9501f574233ae0d7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 2023-02-13 00:54:53,939 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:54:53,940 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 38ms, sequenceid=45, compaction requested=true 2023-02-13 00:54:53,940 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:53,940 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:53,940 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:53,943 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,943 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,943 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,944 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:53,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c124efb[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:53,946 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:53,946 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,948 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:53,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:53,948 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:53,949 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,949 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,949 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,950 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:53,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,950 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:53,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:53,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:53,953 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:53,953 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:53,953 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:53,954 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:53,954 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:53,956 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,956 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,956 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,956 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:53,956 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,957 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,958 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,958 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:53,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,959 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:53,960 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:53,960 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,963 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,963 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:53,963 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:53,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:53,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:53,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:53,963 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:53,964 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,964 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:53,964 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:53,964 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:53,964 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:53,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,064 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:54,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:54,081 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:54,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:54,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:54,083 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:54,083 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,083 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:54,083 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:54,084 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,084 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,085 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,085 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,085 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,086 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,087 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,087 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,087 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:54,087 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:54,087 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,087 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:54,088 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,091 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,091 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,091 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,091 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,091 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,101 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/d42a11e83f7e453fbdd472686f826206 2023-02-13 00:54:54,113 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/bb417d967fb4404d849c0cd5c6f6222c 2023-02-13 00:54:54,118 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/d42a11e83f7e453fbdd472686f826206 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 2023-02-13 00:54:54,122 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:54,123 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/bb417d967fb4404d849c0cd5c6f6222c as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c 2023-02-13 00:54:54,127 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:54:54,127 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 40ms, sequenceid=58, compaction requested=true 2023-02-13 00:54:54,128 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:54,128 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,128 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,129 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,129 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,130 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:54,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36e6a78d[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:54,132 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:54,132 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:54,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,139 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,139 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,140 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:54,142 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:54,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:54,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:54,142 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:54,142 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,145 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,144 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,145 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,146 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,147 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,147 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,148 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,150 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:54,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:54,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,150 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,251 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:54,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:54,267 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:54:54,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:54:54,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 00:54:54,268 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 00:54:54,268 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,268 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 00:54:54,268 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:54:54,269 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,269 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,270 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,270 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,270 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,270 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,271 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 00:54:54,273 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:54:54,272 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,273 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,273 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:54:54,273 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:54:54,273 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:54:54,273 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:54,273 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. started... 2023-02-13 00:54:54,273 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:54:54,275 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,276 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,283 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/6c5c7c1685b84e129cbaf47ddafd882a 2023-02-13 00:54:54,299 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/6c4addb49bc94116931d93b8707ba6c1 2023-02-13 00:54:54,304 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/6c5c7c1685b84e129cbaf47ddafd882a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a 2023-02-13 00:54:54,309 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:54,310 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/6c4addb49bc94116931d93b8707ba6c1 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 2023-02-13 00:54:54,314 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:54:54,315 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 42ms, sequenceid=71, compaction requested=true 2023-02-13 00:54:54,315 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:54,315 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:54,315 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:54:54,315 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:54:54,315 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,315 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 00:54:54,315 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 00:54:54,317 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,317 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,317 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,317 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:54:54,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 00:54:54,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13048230[Count = 0] remaining members to acquire global barrier 2023-02-13 00:54:54,319 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:54:54,319 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,320 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,321 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,321 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:54:54,321 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:54:54,328 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,328 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,328 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,329 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 00:54:54,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:54:54,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:54:54,332 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 00:54:54,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:54:54,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:54:54,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 00:54:54,332 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:54:54,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:54:54,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,334 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:54:54,334 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,335 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,335 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,337 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:54:54,337 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 00:54:54,338 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,345 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:54:54,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 00:54:54,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:54:54,345 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,345 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:54:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,345 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:54:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:54:54,345 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,345 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:54:54,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:54:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 00:54:54,346 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 00:54:54,445 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:54:54,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:54:54,708 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 00:54:54,709 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:54,710 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,711 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,712 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,712 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:54,712 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:54,713 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:54,961 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:54,961 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:54,962 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:54,963 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 00:54:54,964 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:54,964 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:54,964 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:54,964 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:54,965 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:54,965 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:54,965 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:54,965 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:54,965 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:54,966 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:54,966 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:54,966 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:54,966 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:54,966 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,967 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:54,967 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,967 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,967 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,967 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:54,967 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:54,968 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:54,968 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249724968 2023-02-13 00:54:54,970 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,970 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:54,972 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:54,972 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:54,972 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:54,972 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249754972 2023-02-13 00:54:54,973 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249814972 2023-02-13 00:54:54,973 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:54,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:54,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:54,976 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:54,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:54,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:55,223 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:55,223 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:55,224 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:55,224 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:55,224 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:55,225 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 00:54:55,225 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:55,226 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:55,226 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:55,226 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:55,226 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,227 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:55,227 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,227 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,227 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,228 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:55,228 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,228 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,228 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:55,229 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,229 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:55,229 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,229 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,229 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,229 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:55,229 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:55,230 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:55,230 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249725230 2023-02-13 00:54:55,230 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:55,232 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,234 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,234 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,234 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:55,234 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249755234 2023-02-13 00:54:55,234 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249815234 2023-02-13 00:54:55,234 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:55,234 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:55,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN}] 2023-02-13 00:54:55,236 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:55,236 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:55,237 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:55,484 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,485 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:55,485 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:55,485 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:55,486 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:55,486 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:55,486 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:55,486 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:55,486 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:55,487 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:55,488 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 00:54:55,488 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:55,489 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:55,489 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:55,489 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:55,489 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,490 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:55,490 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,490 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,490 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,490 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:55,491 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,491 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,492 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:55,492 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,492 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:55,492 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,492 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,492 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,492 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:55,493 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:55,493 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:55,494 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249725494 2023-02-13 00:54:55,495 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:55,495 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,497 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,497 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,498 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:55,498 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249755498 2023-02-13 00:54:55,498 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249815498 2023-02-13 00:54:55,498 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:55,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN}] 2023-02-13 00:54:55,498 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:55,500 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:55,501 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:55,501 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249695501"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249695501"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249695501"}]},"ts":"1676249695501"} 2023-02-13 00:54:55,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:55,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:55,502 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:55,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:55,502 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:55,748 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,748 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:55,748 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:55,748 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:55,751 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:55,751 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:55,751 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:55,751 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:55,751 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:55,752 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:55,753 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:55,753 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:55,753 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:55,753 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:55,754 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:55,754 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:55,754 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,754 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:55,754 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,755 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:55,755 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,755 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:55,755 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,756 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:55,756 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:55,756 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,757 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:55,757 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:55,757 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,757 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,757 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:55,757 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:55,757 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:55,757 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:55,758 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:55,758 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:55,758 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:55,759 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249725759 2023-02-13 00:54:55,759 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:55,759 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:55,761 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,762 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:55,762 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:55,762 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249755762 2023-02-13 00:54:55,762 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249815762 2023-02-13 00:54:55,762 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:55,762 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:55,762 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249695762"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249695762"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249695762"}]},"ts":"1676249695762"} 2023-02-13 00:54:55,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:55,764 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:55,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:55,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ea3eaa9ea79a3bf34eba60089ea7ec8c, disabling compactions & flushes 2023-02-13 00:54:55,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:55,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:55,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. after waiting 0 ms 2023-02-13 00:54:55,916 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:55,917 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:54:55,927 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 2023-02-13 00:54:55,939 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 2023-02-13 00:54:55,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 2023-02-13 00:54:55,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:55,948 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 2023-02-13 00:54:55,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:54:55,955 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 38ms, sequenceid=82, compaction requested=true 2023-02-13 00:54:55,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:54:55,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:54:55,964 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:55,964 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: Waiting for close lock at 1676249695916Running coprocessor pre-close hooks at 1676249695916Disabling compacts and flushes for region at 1676249695916Disabling writes for close at 1676249695916Obtaining lock to block concurrent updates at 1676249695917 (+1 ms)Preparing flush snapshotting stores in ea3eaa9ea79a3bf34eba60089ea7ec8c at 1676249695917Finished memstore snapshotting testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249695917Flushing stores of testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. at 1676249695917Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1: creating writer at 1676249695918 (+1 ms)Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1: appending metadata at 1676249695920 (+2 ms)Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1: closing flushed file at 1676249695920Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2: creating writer at 1676249695931 (+11 ms)Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2: appending metadata at 1676249695933 (+2 ms)Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2: closing flushed file at 1676249695933Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1: reopening flushed file at 1676249695944 (+11 ms)Flushing ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2: reopening flushed file at 1676249695949 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ea3eaa9ea79a3bf34eba60089ea7ec8c in 38ms, sequenceid=82, compaction requested=true at 1676249695955 (+6 ms)Writing region close event to WAL at 1676249695957 (+2 ms)Running coprocessor post-close hooks at 1676249695964 (+7 ms)Closed at 1676249695964 2023-02-13 00:54:55,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:55,967 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED 2023-02-13 00:54:55,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249695967"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249695967"}]},"ts":"1676249695967"} 2023-02-13 00:54:55,969 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:55,969 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:55,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:55,970 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:56,012 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,012 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:56,012 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:56,012 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:56,013 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:56,014 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:56,014 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:56,014 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:56,014 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:56,015 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:56,015 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:56,016 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 00:54:56,016 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 00:54:56,016 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:56,016 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:56,016 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:56,017 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:56,017 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,017 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:56,017 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,017 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,018 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,018 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:56,018 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,018 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,019 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:56,019 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,019 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,020 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:56,020 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:56,020 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,020 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,020 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,020 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:56,020 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,020 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:56,021 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:56,021 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:56,022 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:56,022 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:56,027 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249726027 2023-02-13 00:54:56,027 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:56,028 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:56,030 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,030 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,030 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:56,030 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249756030 2023-02-13 00:54:56,031 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249816030 2023-02-13 00:54:56,031 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:56,031 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:56,031 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249696031"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249696031"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249696031"}]},"ts":"1676249696031"} 2023-02-13 00:54:56,033 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:56,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 00:54:56,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 269 msec 2023-02-13 00:54:56,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:56,036 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:56,036 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:56,281 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,281 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:56,281 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:56,281 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:56,282 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:56,282 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:56,282 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:56,282 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:56,283 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:56,283 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:56,285 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:56,285 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 00:54:56,285 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:56,286 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:56,286 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:56,286 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:56,287 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:56,287 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,288 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:56,288 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,288 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,288 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,289 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:56,289 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,289 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,289 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:56,290 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,291 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,291 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:56,291 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:56,291 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,291 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,291 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,291 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:56,291 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:56,291 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:54:56,292 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:56,292 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:56,293 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:56,293 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249726293 2023-02-13 00:54:56,294 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,294 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:56,296 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,297 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,297 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:56,297 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249756297 2023-02-13 00:54:56,297 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249816297 2023-02-13 00:54:56,297 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:56,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,299 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:56,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 00:54:56,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN in 801 msec 2023-02-13 00:54:56,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:56,305 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,305 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:56,305 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:56,547 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:56,548 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:56,548 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:56,549 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:56,550 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:56,550 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:56,550 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:56,551 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:56,551 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:56,552 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:56,552 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:56,552 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,553 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:56,553 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,553 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:56,554 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,554 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:56,554 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,554 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:56,555 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:56,556 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 00:54:56,556 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:56,557 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:56,557 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249726557 2023-02-13 00:54:56,559 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:56,559 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,561 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,561 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:56,561 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:56,562 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249756562 2023-02-13 00:54:56,562 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249816562 2023-02-13 00:54:56,562 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:56,562 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,562 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,562 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,567 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:56,575 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, threads=14 2023-02-13 00:54:56,576 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,576 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,576 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,577 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,578 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,578 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,578 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,579 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,579 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,730 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,760 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,769 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,784 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,786 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,787 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,789 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,794 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,794 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,795 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,796 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,797 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,797 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,807 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:56,807 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ea3eaa9ea79a3bf34eba60089ea7ec8c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:54:56,855 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:56,855 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:56,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:56,855 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:57,062 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,062 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:57,062 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:57,062 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:57,063 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:57,063 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:57,063 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:57,063 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:57,063 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:57,064 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:57,065 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 00:54:57,065 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:57,065 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:57,065 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:57,066 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:57,066 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:57,066 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:57,067 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,067 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:57,067 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,067 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,067 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,068 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:57,068 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,068 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,068 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:57,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:57,069 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:57,069 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:57,070 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,070 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,070 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,070 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,070 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,070 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 00:54:57,071 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:57,072 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:57,072 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249727072 2023-02-13 00:54:57,074 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:57,074 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,076 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,076 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,076 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:57,076 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249757076 2023-02-13 00:54:57,076 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249817076 2023-02-13 00:54:57,076 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:57,085 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, threads=14 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,086 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,088 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,088 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,088 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,088 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,088 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,089 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,089 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,089 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,183 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,187 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,190 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,206 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,209 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,230 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,232 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,233 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,236 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,240 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,242 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,244 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,247 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,248 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 for region: ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:57,248 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region ea3eaa9ea79a3bf34eba60089ea7ec8c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:54:57,262 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:54:57,299 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:57,314 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/411efaba7fb06518f9e9fb451db87e25/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:57,316 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/68cbdddc658221d9acd276beb5e2889e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:54:57,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:57,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:57,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:57,327 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:57,327 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:57,328 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:57,329 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 00:54:57,329 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:57,329 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:57,330 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:57,330 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:57,330 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:57,331 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:57,331 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,331 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:57,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,332 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:57,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,333 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:57,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:57,334 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:57,338 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:57,338 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,338 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,338 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,338 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,338 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,338 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 00:54:57,339 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:57,340 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:57,340 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249727340 2023-02-13 00:54:57,340 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:57,341 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,344 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,344 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,344 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:57,344 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249757344 2023-02-13 00:54:57,344 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249817344 2023-02-13 00:54:57,344 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:57,347 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:57,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:57,351 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,351 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:57,351 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:57,595 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:57,595 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:57,596 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:57,596 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:57,597 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 00:54:57,597 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:57,597 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:57,598 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:57,598 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:57,598 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:57,598 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:57,598 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:57,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,599 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,600 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:57,600 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,600 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,600 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:57,601 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,601 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,601 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,601 DEBUG [Listener at localhost/43763] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,601 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:57,602 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:57,602 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:57,603 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249727603 2023-02-13 00:54:57,605 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,606 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:57,608 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,608 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,608 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:57,608 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249757608 2023-02-13 00:54:57,608 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249817608 2023-02-13 00:54:57,608 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:57,609 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:57,620 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:57,620 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:57,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:57,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:57,859 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,859 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:57,859 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:57,860 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:57,860 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:57,860 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:57,860 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:57,860 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:57,860 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:57,861 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:57,862 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 00:54:57,862 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:57,862 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:57,862 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:57,863 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:57,863 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:57,863 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:57,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:57,864 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,865 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:57,865 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,865 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:57,865 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,866 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:57,866 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:57,867 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,867 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:57,867 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:57,868 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:57,868 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,868 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,868 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,868 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,868 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:57,868 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:54:57,869 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:57,869 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:57,870 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249727870 2023-02-13 00:54:57,870 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:57,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:57,876 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,876 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:57,876 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:57,876 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249757876 2023-02-13 00:54:57,876 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249817876 2023-02-13 00:54:57,876 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 00:54:57,877 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:57,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:57,879 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:57,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:57,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:58,127 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:58,128 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:58,128 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:58,129 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:58,130 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 00:54:58,130 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:58,130 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:58,130 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:58,130 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:58,131 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:58,131 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:58,131 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,131 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:58,131 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,132 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,132 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,132 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:58,132 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,132 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,133 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:58,133 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,134 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:58,134 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:58,134 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:58,134 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,134 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,134 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,134 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,134 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,134 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:58,135 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:58,135 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:58,135 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249728135 2023-02-13 00:54:58,136 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:58,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,139 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,139 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,139 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:58,139 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249758139 2023-02-13 00:54:58,140 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249818140 2023-02-13 00:54:58,140 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:58,140 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:58,142 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:58,142 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:58,142 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:58,142 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:58,390 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,390 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:58,390 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:58,390 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:58,391 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:58,391 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:58,391 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:58,391 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:58,391 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:58,392 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:58,393 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 00:54:58,393 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:58,393 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:58,393 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:58,393 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,394 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,395 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,395 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:58,395 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,395 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,395 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:58,396 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,396 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,396 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,396 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,397 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:58,397 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:58,397 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:58,398 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249728398 2023-02-13 00:54:58,398 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:58,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,401 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,402 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,402 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:58,402 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249758402 2023-02-13 00:54:58,402 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249818402 2023-02-13 00:54:58,402 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:58,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:58,406 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN] 2023-02-13 00:54:58,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:58,406 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:58,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:58,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:58,652 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,652 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:58,652 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:58,652 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:58,653 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:58,653 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:58,653 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:58,654 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:58,654 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:58,654 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:58,655 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 00:54:58,655 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:54:58,655 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:54:58,655 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:58,655 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:58,656 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:58,656 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:58,656 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:58,656 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,656 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,657 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,658 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:58,658 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:58,659 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,659 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,659 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,659 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,659 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:58,660 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:58,660 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:58,660 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:58,661 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249728661 2023-02-13 00:54:58,661 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:58,663 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,663 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:58,665 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,665 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,665 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:58,665 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249758665 2023-02-13 00:54:58,665 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249818665 2023-02-13 00:54:58,666 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:58,666 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:58,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:58,670 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:58,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:58,670 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:58,916 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,916 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:58,916 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:58,916 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:58,917 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:58,917 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:58,917 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:58,917 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:58,917 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:58,918 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:58,919 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:54:58,919 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:58,919 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:58,920 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:58,921 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:58,922 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,922 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:58,922 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:58,922 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,922 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,922 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,922 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:58,922 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 00:54:58,923 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:58,923 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:58,923 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:58,924 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249728924 2023-02-13 00:54:58,924 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:58,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:58,926 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:58,928 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,928 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:58,928 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:58,929 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249758928 2023-02-13 00:54:58,929 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249818929 2023-02-13 00:54:58,929 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:54:58,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:58,931 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:58,931 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:58,931 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:58,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:59,179 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,179 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:59,179 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:59,179 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:59,181 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:59,182 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:59,182 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:59,182 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:59,182 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:59,182 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:59,183 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 00:54:59,183 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:59,183 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:59,184 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:59,184 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:59,184 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:59,185 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,185 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:59,185 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,185 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,186 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,186 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:59,186 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,187 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,187 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:59,187 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,188 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,188 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:59,188 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,188 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,188 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,188 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:59,188 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:59,189 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:59,189 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:59,189 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:59,190 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249729190 2023-02-13 00:54:59,190 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:59,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,192 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,194 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,194 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,194 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:59,194 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249759194 2023-02-13 00:54:59,194 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249819194 2023-02-13 00:54:59,194 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:54:59,195 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:59,196 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:59,196 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 00:54:59,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:59,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:59,445 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:59,445 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:59,446 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:59,447 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 00:54:59,447 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 00:54:59,447 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:59,447 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:59,448 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:59,448 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:59,448 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,449 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:59,449 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,449 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,449 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,450 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:59,450 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,450 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,450 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:59,451 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,451 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,451 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:59,451 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,451 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,451 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,451 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:54:59,452 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 00:54:59,452 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:59,453 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:59,453 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:59,453 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249729453 2023-02-13 00:54:59,454 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:59,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,457 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,459 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,460 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,460 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:59,460 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249759460 2023-02-13 00:54:59,460 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249819460 2023-02-13 00:54:59,460 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 00:54:59,460 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:54:59,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e exec-time=4.7510 sec 2023-02-13 00:54:59,463 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,464 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=true 2023-02-13 00:54:59,614 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:54:59,615 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:59,615 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249699615"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249699615"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249699615"}]},"ts":"1676249699615"} 2023-02-13 00:54:59,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:59,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:54:59,617 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:54:59,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:54:59,710 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:54:59,710 WARN [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 00:54:59,710 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:54:59,710 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:54:59,710 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:54:59,711 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:54:59,711 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:54:59,711 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:54:59,711 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:54:59,711 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:54:59,711 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:54:59,712 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 00:54:59,712 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:54:59,713 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:54:59,713 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:54:59,713 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:54:59,713 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,713 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,714 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:54:59,715 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:54:59,715 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:54:59,715 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,715 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:54:59,715 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,715 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-13 00:54:59,716 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:54:59,716 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:54:59,716 INFO [Listener at localhost/43763] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c to restore RIT 2023-02-13 00:54:59,717 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249729717 2023-02-13 00:54:59,718 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:54:59,718 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:54:59,719 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,720 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:54:59,720 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:54:59,720 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249759720 2023-02-13 00:54:59,720 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249819720 2023-02-13 00:54:59,720 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:54:59,720 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:59,720 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249699720"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249699720"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249699720"}]},"ts":"1676249699720"} 2023-02-13 00:54:59,722 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:54:59,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:59,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ea3eaa9ea79a3bf34eba60089ea7ec8c, NAME => 'testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:54:59,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:54:59,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:54:59,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,877 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,879 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,879 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 2023-02-13 00:54:59,880 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 2023-02-13 00:54:59,880 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea3eaa9ea79a3bf34eba60089ea7ec8c columnFamilyName cf1 2023-02-13 00:54:59,886 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c 2023-02-13 00:54:59,889 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 2023-02-13 00:54:59,892 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a 2023-02-13 00:54:59,895 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 2023-02-13 00:54:59,898 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 2023-02-13 00:54:59,901 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 2023-02-13 00:54:59,904 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 2023-02-13 00:54:59,905 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(310): Store=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:59,905 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,905 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 2023-02-13 00:54:59,906 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 2023-02-13 00:54:59,906 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea3eaa9ea79a3bf34eba60089ea7ec8c columnFamilyName cf2 2023-02-13 00:54:59,911 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 2023-02-13 00:54:59,914 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 2023-02-13 00:54:59,917 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 2023-02-13 00:54:59,920 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 2023-02-13 00:54:59,924 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 2023-02-13 00:54:59,926 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c 2023-02-13 00:54:59,930 DEBUG [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 2023-02-13 00:54:59,930 INFO [StoreOpener-ea3eaa9ea79a3bf34eba60089ea7ec8c-1] regionserver.HStore(310): Store=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:54:59,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.splits 2023-02-13 00:54:59,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:54:59,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:54:59,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ea3eaa9ea79a3bf34eba60089ea7ec8c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61779570, jitterRate=-0.07941266894340515}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:54:59,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: Running coprocessor pre-open hook at 1676249699877Writing region info on filesystem at 1676249699877Initializing all the Stores at 1676249699878 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249699878Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249699878Cleaning up temporary data from old regions at 1676249699932 (+54 ms)Cleaning up detritus from prior splits at 1676249699933 (+1 ms)Running coprocessor post-open hooks at 1676249699936 (+3 ms)Region opened successfully at 1676249699936 2023-02-13 00:54:59,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c., pid=48, masterSystemTime=1676249699874 2023-02-13 00:54:59,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:59,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:54:59,938 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:59,938 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:54:59,939 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 00:54:59,939 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 00:54:59,939 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 is initiating minor compaction (all files) 2023-02-13 00:54:59,939 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 is initiating minor compaction (all files) 2023-02-13 00:54:59,940 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 in testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:59,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:59,940 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 in testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:59,940 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp, totalSize=34.9 K 2023-02-13 00:54:59,940 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp, totalSize=34.9 K 2023-02-13 00:54:59,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:54:59,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:54:59,940 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 7e142745db734ace983d997d66aea3f5, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249693284 2023-02-13 00:54:59,940 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 170ffcfbdec445d0813fcf7c348fd80c, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249693284 2023-02-13 00:54:59,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249699940"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249699940"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249699940"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249699940"}]},"ts":"1676249699940"} 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 8e11cbb1a8004c50ad90cb28e9961614, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249693465 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 70e6217dd7ce4a95924e683694741f00, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249693465 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting bb605aa7640b48c691be8526e9468ed5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249693682 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 5de7c3e4e9074f42affca8eb62a9bce1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249693682 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 39375d12364d4a7e9501f574233ae0d7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249693882 2023-02-13 00:54:59,941 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting fba7d27e3e13454abf830a904102c1f4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249693882 2023-02-13 00:54:59,942 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting bb417d967fb4404d849c0cd5c6f6222c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249694066 2023-02-13 00:54:59,942 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting d42a11e83f7e453fbdd472686f826206, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249694066 2023-02-13 00:54:59,942 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 6c4addb49bc94116931d93b8707ba6c1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249694253 2023-02-13 00:54:59,943 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 6c5c7c1685b84e129cbaf47ddafd882a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249694253 2023-02-13 00:54:59,943 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 70ca1802d22446ac935b5fbd0ec5fdc6, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249694447 2023-02-13 00:54:59,943 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting a02c2ac8f3e649ce83b96f92d58ba8f5, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249694447 2023-02-13 00:54:59,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 00:54:59,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 221 msec 2023-02-13 00:54:59,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN in 1.5410 sec 2023-02-13 00:54:59,963 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): ea3eaa9ea79a3bf34eba60089ea7ec8c#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:59,964 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): ea3eaa9ea79a3bf34eba60089ea7ec8c#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:54:59,981 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf2/52622f1531e045988bee918100f561a6 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/52622f1531e045988bee918100f561a6 2023-02-13 00:54:59,983 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/.tmp/cf1/a52edca893b94a7889419e0cf62911e9 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a52edca893b94a7889419e0cf62911e9 2023-02-13 00:54:59,986 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2 of ea3eaa9ea79a3bf34eba60089ea7ec8c into 52622f1531e045988bee918100f561a6(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:59,986 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:59,986 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c., storeName=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2, priority=9, startTime=1676249699938; duration=0sec 2023-02-13 00:54:59,987 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:54:59,989 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1 of ea3eaa9ea79a3bf34eba60089ea7ec8c into a52edca893b94a7889419e0cf62911e9(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:54:59,989 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: 2023-02-13 00:54:59,989 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c., storeName=ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1, priority=9, startTime=1676249699937; duration=0sec 2023-02-13 00:54:59,989 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:00,297 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 00:55:00,299 DEBUG [Listener at localhost/43763] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 00:55:00,301 INFO [Listener at localhost/43763] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 00:55:00,303 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 00:55:00,303 INFO [Listener at localhost/43763] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 00:55:00,304 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 00:55:00,319 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:00,320 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 00:55:00,321 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-13 00:55:00,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 00:55:00,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249700324"}]},"ts":"1676249700324"} 2023-02-13 00:55:00,326 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 00:55:00,327 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 00:55:00,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN}] 2023-02-13 00:55:00,328 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN 2023-02-13 00:55:00,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:00,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249700329"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249700329"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249700329"}]},"ts":"1676249700329"} 2023-02-13 00:55:00,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:00,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 00:55:00,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:55:00,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ea3eaa9ea79a3bf34eba60089ea7ec8c, disabling compactions & flushes 2023-02-13 00:55:00,482 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:55:00,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:55:00,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. after waiting 0 ms 2023-02-13 00:55:00,482 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:55:00,483 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5] to archive 2023-02-13 00:55:00,484 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:00,486 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/170ffcfbdec445d0813fcf7c348fd80c 2023-02-13 00:55:00,487 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/70e6217dd7ce4a95924e683694741f00 2023-02-13 00:55:00,488 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/5de7c3e4e9074f42affca8eb62a9bce1 2023-02-13 00:55:00,490 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/fba7d27e3e13454abf830a904102c1f4 2023-02-13 00:55:00,491 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/d42a11e83f7e453fbdd472686f826206 2023-02-13 00:55:00,493 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/6c5c7c1685b84e129cbaf47ddafd882a 2023-02-13 00:55:00,494 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a02c2ac8f3e649ce83b96f92d58ba8f5 2023-02-13 00:55:00,499 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6] to archive 2023-02-13 00:55:00,500 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:00,501 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/7e142745db734ace983d997d66aea3f5 2023-02-13 00:55:00,502 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/8e11cbb1a8004c50ad90cb28e9961614 2023-02-13 00:55:00,503 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb605aa7640b48c691be8526e9468ed5 2023-02-13 00:55:00,504 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/39375d12364d4a7e9501f574233ae0d7 2023-02-13 00:55:00,505 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/bb417d967fb4404d849c0cd5c6f6222c 2023-02-13 00:55:00,506 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/6c4addb49bc94116931d93b8707ba6c1 2023-02-13 00:55:00,508 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/70ca1802d22446ac935b5fbd0ec5fdc6 2023-02-13 00:55:00,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:55:00,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:00,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c. 2023-02-13 00:55:00,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ea3eaa9ea79a3bf34eba60089ea7ec8c: Waiting for close lock at 1676249700482Running coprocessor pre-close hooks at 1676249700482Disabling compacts and flushes for region at 1676249700482Disabling writes for close at 1676249700482Writing region close event to WAL at 1676249700509 (+27 ms)Running coprocessor post-close hooks at 1676249700512 (+3 ms)Closed at 1676249700512 2023-02-13 00:55:00,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:55:00,515 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=ea3eaa9ea79a3bf34eba60089ea7ec8c, regionState=CLOSED 2023-02-13 00:55:00,515 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249700515"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249700515"}]},"ts":"1676249700515"} 2023-02-13 00:55:00,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 00:55:00,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure ea3eaa9ea79a3bf34eba60089ea7ec8c, server=jenkins-hbase4.apache.org,41539,1676249678442 in 186 msec 2023-02-13 00:55:00,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 00:55:00,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, UNASSIGN in 191 msec 2023-02-13 00:55:00,521 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249700521"}]},"ts":"1676249700521"} 2023-02-13 00:55:00,522 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 00:55:00,524 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 00:55:00,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 203 msec 2023-02-13 00:55:00,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 00:55:00,627 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 00:55:00,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-13 00:55:00,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,630 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,631 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 00:55:00,634 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:55:00,636 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits] 2023-02-13 00:55:00,638 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a52edca893b94a7889419e0cf62911e9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf1/a52edca893b94a7889419e0cf62911e9 2023-02-13 00:55:00,640 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/52622f1531e045988bee918100f561a6 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/cf2/52622f1531e045988bee918100f561a6 2023-02-13 00:55:00,643 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c/recovered.edits/90.seqid 2023-02-13 00:55:00,644 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackAndDoubleExecution/ea3eaa9ea79a3bf34eba60089ea7ec8c 2023-02-13 00:55:00,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 00:55:00,648 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,650 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 00:55:00,652 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 00:55:00,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 00:55:00,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249700653"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:00,655 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 00:55:00,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ea3eaa9ea79a3bf34eba60089ea7ec8c, NAME => 'testRollbackAndDoubleExecution,,1676249692779.ea3eaa9ea79a3bf34eba60089ea7ec8c.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 00:55:00,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 00:55:00,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249700655"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:00,657 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 00:55:00,664 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:55:00,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 36 msec 2023-02-13 00:55:00,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 00:55:00,733 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 00:55:00,747 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=393 (was 396), OpenFileDescriptor=858 (was 858), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=233 (was 254), ProcessCount=167 (was 167), AvailableMemoryMB=3499 (was 3607) 2023-02-13 00:55:00,761 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=393, OpenFileDescriptor=858, MaxFileDescriptor=60000, SystemLoadAverage=223, ProcessCount=167, AvailableMemoryMB=3499 2023-02-13 00:55:00,761 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:00,765 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:55:00,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:55:00,767 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:00,768 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:00,769 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:00,770 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:00,771 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 empty. 2023-02-13 00:55:00,771 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:00,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 00:55:00,788 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:00,790 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 3bb3523ffc664edd904a25ce35edbcb3, NAME => 'testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 3bb3523ffc664edd904a25ce35edbcb3, disabling compactions & flushes 2023-02-13 00:55:00,804 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. after waiting 0 ms 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:00,804 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:00,804 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 3bb3523ffc664edd904a25ce35edbcb3: Waiting for close lock at 1676249700804Disabling compacts and flushes for region at 1676249700804Disabling writes for close at 1676249700804Writing region close event to WAL at 1676249700804Closed at 1676249700804 2023-02-13 00:55:00,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:00,807 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249700807"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249700807"}]},"ts":"1676249700807"} 2023-02-13 00:55:00,809 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:55:00,810 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:00,810 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249700810"}]},"ts":"1676249700810"} 2023-02-13 00:55:00,811 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 00:55:00,814 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:55:00,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, ASSIGN}] 2023-02-13 00:55:00,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, ASSIGN 2023-02-13 00:55:00,818 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:55:00,968 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:55:00,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=3bb3523ffc664edd904a25ce35edbcb3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:00,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249700969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249700969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249700969"}]},"ts":"1676249700969"} 2023-02-13 00:55:00,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 3bb3523ffc664edd904a25ce35edbcb3, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:01,123 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,123 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:55:01,126 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48482, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:55:01,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3bb3523ffc664edd904a25ce35edbcb3, NAME => 'testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:01,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,133 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,135 DEBUG [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf1 2023-02-13 00:55:01,135 DEBUG [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf1 2023-02-13 00:55:01,135 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bb3523ffc664edd904a25ce35edbcb3 columnFamilyName cf1 2023-02-13 00:55:01,136 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] regionserver.HStore(310): Store=3bb3523ffc664edd904a25ce35edbcb3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,136 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,137 DEBUG [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf2 2023-02-13 00:55:01,137 DEBUG [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf2 2023-02-13 00:55:01,138 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3bb3523ffc664edd904a25ce35edbcb3 columnFamilyName cf2 2023-02-13 00:55:01,138 INFO [StoreOpener-3bb3523ffc664edd904a25ce35edbcb3-1] regionserver.HStore(310): Store=3bb3523ffc664edd904a25ce35edbcb3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:01,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:01,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3bb3523ffc664edd904a25ce35edbcb3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75281023, jitterRate=0.12177465856075287}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:01,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3bb3523ffc664edd904a25ce35edbcb3: Running coprocessor pre-open hook at 1676249701132Writing region info on filesystem at 1676249701132Initializing all the Stores at 1676249701133 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701133Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701133Cleaning up temporary data from old regions at 1676249701140 (+7 ms)Cleaning up detritus from prior splits at 1676249701141 (+1 ms)Running coprocessor post-open hooks at 1676249701146 (+5 ms)Region opened successfully at 1676249701146 2023-02-13 00:55:01,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3., pid=55, masterSystemTime=1676249701123 2023-02-13 00:55:01,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,150 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=3bb3523ffc664edd904a25ce35edbcb3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,151 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249701150"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249701150"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249701150"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249701150"}]},"ts":"1676249701150"} 2023-02-13 00:55:01,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 00:55:01,153 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 3bb3523ffc664edd904a25ce35edbcb3, server=jenkins-hbase4.apache.org,34557,1676249678414 in 181 msec 2023-02-13 00:55:01,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 00:55:01,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, ASSIGN in 339 msec 2023-02-13 00:55:01,156 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:01,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249701156"}]},"ts":"1676249701156"} 2023-02-13 00:55:01,157 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 00:55:01,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:01,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 393 msec 2023-02-13 00:55:01,268 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,269 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, UNASSIGN}] 2023-02-13 00:55:01,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, UNASSIGN 2023-02-13 00:55:01,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=3bb3523ffc664edd904a25ce35edbcb3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249701273"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249701273"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249701273"}]},"ts":"1676249701273"} 2023-02-13 00:55:01,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 3bb3523ffc664edd904a25ce35edbcb3, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:01,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3bb3523ffc664edd904a25ce35edbcb3, disabling compactions & flushes 2023-02-13 00:55:01,428 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. after waiting 0 ms 2023-02-13 00:55:01,428 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 00:55:01,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:01,433 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3. 2023-02-13 00:55:01,433 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3bb3523ffc664edd904a25ce35edbcb3: Waiting for close lock at 1676249701427Running coprocessor pre-close hooks at 1676249701427Disabling compacts and flushes for region at 1676249701427Disabling writes for close at 1676249701428 (+1 ms)Writing region close event to WAL at 1676249701429 (+1 ms)Running coprocessor post-close hooks at 1676249701433 (+4 ms)Closed at 1676249701433 2023-02-13 00:55:01,435 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:01,435 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=3bb3523ffc664edd904a25ce35edbcb3, regionState=CLOSED 2023-02-13 00:55:01,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249701435"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249701435"}]},"ts":"1676249701435"} 2023-02-13 00:55:01,438 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 00:55:01,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 3bb3523ffc664edd904a25ce35edbcb3, server=jenkins-hbase4.apache.org,34557,1676249678414 in 162 msec 2023-02-13 00:55:01,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 00:55:01,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=3bb3523ffc664edd904a25ce35edbcb3, UNASSIGN in 167 msec 2023-02-13 00:55:01,478 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 00:55:01,480 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 00:55:01,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676249701482"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676249701482"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676249701482"}]},"ts":"1676249701482"} 2023-02-13 00:55:01,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249701482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249701482"}]},"ts":"1676249701482"} 2023-02-13 00:55:01,482 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701482"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249701482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249701482"}]},"ts":"1676249701482"} 2023-02-13 00:55:01,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, ASSIGN}] 2023-02-13 00:55:01,486 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, ASSIGN 2023-02-13 00:55:01,487 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:55:01,488 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, ASSIGN 2023-02-13 00:55:01,489 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:55:01,637 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:55:01,639 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=35ea1bdc294f2a77cb80ecfd357e548e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,639 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701639"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249701639"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249701639"}]},"ts":"1676249701639"} 2023-02-13 00:55:01,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 35ea1bdc294f2a77cb80ecfd357e548e, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:01,644 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=5081bfdc6ad0313a915da5f3b4be3a2e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,644 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701644"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249701644"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249701644"}]},"ts":"1676249701644"} 2023-02-13 00:55:01,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 5081bfdc6ad0313a915da5f3b4be3a2e, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:01,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:01,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5081bfdc6ad0313a915da5f3b4be3a2e, NAME => 'testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 00:55:01,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:01,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:01,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,798 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,799 DEBUG [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf1 2023-02-13 00:55:01,799 DEBUG [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf1 2023-02-13 00:55:01,800 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5081bfdc6ad0313a915da5f3b4be3a2e columnFamilyName cf1 2023-02-13 00:55:01,800 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] regionserver.HStore(310): Store=5081bfdc6ad0313a915da5f3b4be3a2e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,800 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,802 DEBUG [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf2 2023-02-13 00:55:01,802 DEBUG [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf2 2023-02-13 00:55:01,802 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5081bfdc6ad0313a915da5f3b4be3a2e columnFamilyName cf2 2023-02-13 00:55:01,802 INFO [StoreOpener-5081bfdc6ad0313a915da5f3b4be3a2e-1] regionserver.HStore(310): Store=5081bfdc6ad0313a915da5f3b4be3a2e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:01,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:01,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5081bfdc6ad0313a915da5f3b4be3a2e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63712757, jitterRate=-0.05060593783855438}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:01,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5081bfdc6ad0313a915da5f3b4be3a2e: Running coprocessor pre-open hook at 1676249701797Writing region info on filesystem at 1676249701797Initializing all the Stores at 1676249701798 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701798Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701798Cleaning up temporary data from old regions at 1676249701805 (+7 ms)Cleaning up detritus from prior splits at 1676249701805Running coprocessor post-open hooks at 1676249701807 (+2 ms)Region opened successfully at 1676249701807 2023-02-13 00:55:01,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e., pid=62, masterSystemTime=1676249701793 2023-02-13 00:55:01,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:01,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:01,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:01,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35ea1bdc294f2a77cb80ecfd357e548e, NAME => 'testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 00:55:01,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=5081bfdc6ad0313a915da5f3b4be3a2e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:01,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701810"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249701810"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249701810"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249701810"}]},"ts":"1676249701810"} 2023-02-13 00:55:01,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:01,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,812 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,813 DEBUG [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf1 2023-02-13 00:55:01,813 DEBUG [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf1 2023-02-13 00:55:01,813 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35ea1bdc294f2a77cb80ecfd357e548e columnFamilyName cf1 2023-02-13 00:55:01,814 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] regionserver.HStore(310): Store=35ea1bdc294f2a77cb80ecfd357e548e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,814 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,815 DEBUG [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf2 2023-02-13 00:55:01,815 DEBUG [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf2 2023-02-13 00:55:01,815 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35ea1bdc294f2a77cb80ecfd357e548e columnFamilyName cf2 2023-02-13 00:55:01,816 INFO [StoreOpener-35ea1bdc294f2a77cb80ecfd357e548e-1] regionserver.HStore(310): Store=35ea1bdc294f2a77cb80ecfd357e548e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:01,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-13 00:55:01,817 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 5081bfdc6ad0313a915da5f3b4be3a2e, server=jenkins-hbase4.apache.org,34557,1676249678414 in 167 msec 2023-02-13 00:55:01,818 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, ASSIGN in 331 msec 2023-02-13 00:55:01,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:01,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:01,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 35ea1bdc294f2a77cb80ecfd357e548e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66912680, jitterRate=-0.0029233694076538086}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:01,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 35ea1bdc294f2a77cb80ecfd357e548e: Running coprocessor pre-open hook at 1676249701811Writing region info on filesystem at 1676249701811Initializing all the Stores at 1676249701811Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701811Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249701812 (+1 ms)Cleaning up temporary data from old regions at 1676249701819 (+7 ms)Cleaning up detritus from prior splits at 1676249701819Running coprocessor post-open hooks at 1676249701821 (+2 ms)Region opened successfully at 1676249701821 2023-02-13 00:55:01,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e., pid=61, masterSystemTime=1676249701793 2023-02-13 00:55:01,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:01,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:01,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=35ea1bdc294f2a77cb80ecfd357e548e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:01,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249701823"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249701823"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249701823"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249701823"}]},"ts":"1676249701823"} 2023-02-13 00:55:01,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-13 00:55:01,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 35ea1bdc294f2a77cb80ecfd357e548e, server=jenkins-hbase4.apache.org,34557,1676249678414 in 184 msec 2023-02-13 00:55:01,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-13 00:55:01,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, ASSIGN in 341 msec 2023-02-13 00:55:01,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e in 560 msec 2023-02-13 00:55:02,022 DEBUG [hconnection-0x5adffcfe-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 00:55:02,026 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48492, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 00:55:02,027 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:55:02,030 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:02,032 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,032 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 00:55:02,036 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249702036"}]},"ts":"1676249702036"} 2023-02-13 00:55:02,037 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 00:55:02,044 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 00:55:02,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, UNASSIGN}] 2023-02-13 00:55:02,045 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, UNASSIGN 2023-02-13 00:55:02,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=35ea1bdc294f2a77cb80ecfd357e548e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:02,046 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249702046"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249702046"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249702046"}]},"ts":"1676249702046"} 2023-02-13 00:55:02,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 35ea1bdc294f2a77cb80ecfd357e548e, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:02,049 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, UNASSIGN 2023-02-13 00:55:02,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=5081bfdc6ad0313a915da5f3b4be3a2e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:02,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249702050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249702050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249702050"}]},"ts":"1676249702050"} 2023-02-13 00:55:02,051 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 5081bfdc6ad0313a915da5f3b4be3a2e, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:02,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 00:55:02,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:02,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5081bfdc6ad0313a915da5f3b4be3a2e, disabling compactions & flushes 2023-02-13 00:55:02,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:02,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:02,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. after waiting 0 ms 2023-02-13 00:55:02,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:02,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 00:55:02,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:02,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e. 2023-02-13 00:55:02,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5081bfdc6ad0313a915da5f3b4be3a2e: Waiting for close lock at 1676249702199Running coprocessor pre-close hooks at 1676249702199Disabling compacts and flushes for region at 1676249702199Disabling writes for close at 1676249702200 (+1 ms)Writing region close event to WAL at 1676249702200Running coprocessor post-close hooks at 1676249702204 (+4 ms)Closed at 1676249702204 2023-02-13 00:55:02,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:02,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:02,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 35ea1bdc294f2a77cb80ecfd357e548e, disabling compactions & flushes 2023-02-13 00:55:02,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:02,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:02,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. after waiting 0 ms 2023-02-13 00:55:02,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:02,207 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=5081bfdc6ad0313a915da5f3b4be3a2e, regionState=CLOSED 2023-02-13 00:55:02,207 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249702207"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249702207"}]},"ts":"1676249702207"} 2023-02-13 00:55:02,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 00:55:02,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 00:55:02,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 5081bfdc6ad0313a915da5f3b4be3a2e, server=jenkins-hbase4.apache.org,34557,1676249678414 in 157 msec 2023-02-13 00:55:02,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:02,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e. 2023-02-13 00:55:02,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 35ea1bdc294f2a77cb80ecfd357e548e: Waiting for close lock at 1676249702206Running coprocessor pre-close hooks at 1676249702206Disabling compacts and flushes for region at 1676249702206Disabling writes for close at 1676249702207 (+1 ms)Writing region close event to WAL at 1676249702207Running coprocessor post-close hooks at 1676249702210 (+3 ms)Closed at 1676249702210 2023-02-13 00:55:02,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=5081bfdc6ad0313a915da5f3b4be3a2e, UNASSIGN in 166 msec 2023-02-13 00:55:02,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:02,212 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=35ea1bdc294f2a77cb80ecfd357e548e, regionState=CLOSED 2023-02-13 00:55:02,212 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249702212"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249702212"}]},"ts":"1676249702212"} 2023-02-13 00:55:02,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 00:55:02,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 35ea1bdc294f2a77cb80ecfd357e548e, server=jenkins-hbase4.apache.org,34557,1676249678414 in 166 msec 2023-02-13 00:55:02,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-02-13 00:55:02,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=35ea1bdc294f2a77cb80ecfd357e548e, UNASSIGN in 170 msec 2023-02-13 00:55:02,216 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249702216"}]},"ts":"1676249702216"} 2023-02-13 00:55:02,217 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 00:55:02,219 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 00:55:02,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 186 msec 2023-02-13 00:55:02,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 00:55:02,338 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 00:55:02,339 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,341 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,342 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 00:55:02,348 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:02,348 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:02,348 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:02,350 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/recovered.edits] 2023-02-13 00:55:02,350 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/recovered.edits] 2023-02-13 00:55:02,351 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/recovered.edits] 2023-02-13 00:55:02,357 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/recovered.edits/7.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e/recovered.edits/7.seqid 2023-02-13 00:55:02,357 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/recovered.edits/7.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e/recovered.edits/7.seqid 2023-02-13 00:55:02,358 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/5081bfdc6ad0313a915da5f3b4be3a2e 2023-02-13 00:55:02,358 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:55:02,358 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/recovered.edits/4.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3/recovered.edits/4.seqid 2023-02-13 00:55:02,359 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionNoStoreFile/3bb3523ffc664edd904a25ce35edbcb3 2023-02-13 00:55:02,359 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 00:55:02,361 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,363 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 00:55:02,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 00:55:02,366 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,366 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 00:55:02,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249702366"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:02,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249702366"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:02,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249702366"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:02,368 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:55:02,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3bb3523ffc664edd904a25ce35edbcb3, NAME => 'testSplitTableRegionNoStoreFile,,1676249700765.3bb3523ffc664edd904a25ce35edbcb3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5081bfdc6ad0313a915da5f3b4be3a2e, NAME => 'testSplitTableRegionNoStoreFile,,1676249701268.5081bfdc6ad0313a915da5f3b4be3a2e.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 35ea1bdc294f2a77cb80ecfd357e548e, NAME => 'testSplitTableRegionNoStoreFile,41,1676249701268.35ea1bdc294f2a77cb80ecfd357e548e.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 00:55:02,368 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 00:55:02,368 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249702368"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:02,369 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 00:55:02,371 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:55:02,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 32 msec 2023-02-13 00:55:02,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 00:55:02,444 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 00:55:02,462 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=403 (was 393) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-471442127_15 at /127.0.0.1:46226 [Waiting for operation #44] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-471442127_15 at /127.0.0.1:38972 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=861 (was 858) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=223 (was 223), ProcessCount=167 (was 167), AvailableMemoryMB=3492 (was 3499) 2023-02-13 00:55:02,476 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=403, OpenFileDescriptor=861, MaxFileDescriptor=60000, SystemLoadAverage=223, ProcessCount=167, AvailableMemoryMB=3492 2023-02-13 00:55:02,477 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:02,479 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:55:02,479 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:55:02,481 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,482 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:02,483 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:02,485 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,485 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 empty. 2023-02-13 00:55:02,486 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,486 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 00:55:02,497 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:02,498 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => ae67fb354b1c30a4bea3ba0a8e540f12, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing ae67fb354b1c30a4bea3ba0a8e540f12, disabling compactions & flushes 2023-02-13 00:55:02,505 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. after waiting 0 ms 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,505 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,505 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for ae67fb354b1c30a4bea3ba0a8e540f12: Waiting for close lock at 1676249702505Disabling compacts and flushes for region at 1676249702505Disabling writes for close at 1676249702505Writing region close event to WAL at 1676249702505Closed at 1676249702505 2023-02-13 00:55:02,507 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:02,508 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249702508"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249702508"}]},"ts":"1676249702508"} 2023-02-13 00:55:02,509 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:55:02,510 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:02,510 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249702510"}]},"ts":"1676249702510"} 2023-02-13 00:55:02,511 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 00:55:02,514 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:55:02,515 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:55:02,515 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:55:02,515 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:55:02,515 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:55:02,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, ASSIGN}] 2023-02-13 00:55:02,516 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, ASSIGN 2023-02-13 00:55:02,517 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:55:02,667 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:55:02,668 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ae67fb354b1c30a4bea3ba0a8e540f12, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:02,668 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249702668"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249702668"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249702668"}]},"ts":"1676249702668"} 2023-02-13 00:55:02,670 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure ae67fb354b1c30a4bea3ba0a8e540f12, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:02,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ae67fb354b1c30a4bea3ba0a8e540f12, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:02,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:02,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:02,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,826 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,827 DEBUG [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1 2023-02-13 00:55:02,828 DEBUG [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1 2023-02-13 00:55:02,828 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae67fb354b1c30a4bea3ba0a8e540f12 columnFamilyName cf1 2023-02-13 00:55:02,828 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] regionserver.HStore(310): Store=ae67fb354b1c30a4bea3ba0a8e540f12/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:02,828 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,830 DEBUG [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2 2023-02-13 00:55:02,830 DEBUG [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2 2023-02-13 00:55:02,830 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae67fb354b1c30a4bea3ba0a8e540f12 columnFamilyName cf2 2023-02-13 00:55:02,830 INFO [StoreOpener-ae67fb354b1c30a4bea3ba0a8e540f12-1] regionserver.HStore(310): Store=ae67fb354b1c30a4bea3ba0a8e540f12/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:02,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:02,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:02,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:02,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ae67fb354b1c30a4bea3ba0a8e540f12; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71206771, jitterRate=0.06106357276439667}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:02,840 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ae67fb354b1c30a4bea3ba0a8e540f12: Running coprocessor pre-open hook at 1676249702825Writing region info on filesystem at 1676249702825Initializing all the Stores at 1676249702826 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249702826Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249702826Cleaning up temporary data from old regions at 1676249702832 (+6 ms)Cleaning up detritus from prior splits at 1676249702832Running coprocessor post-open hooks at 1676249702840 (+8 ms)Region opened successfully at 1676249702840 2023-02-13 00:55:02,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12., pid=71, masterSystemTime=1676249702822 2023-02-13 00:55:02,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ae67fb354b1c30a4bea3ba0a8e540f12, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:02,843 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249702842"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249702842"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249702842"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249702842"}]},"ts":"1676249702842"} 2023-02-13 00:55:02,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 00:55:02,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure ae67fb354b1c30a4bea3ba0a8e540f12, server=jenkins-hbase4.apache.org,41539,1676249678442 in 174 msec 2023-02-13 00:55:02,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 00:55:02,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, ASSIGN in 331 msec 2023-02-13 00:55:02,848 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:02,848 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249702848"}]},"ts":"1676249702848"} 2023-02-13 00:55:02,849 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 00:55:02,852 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:02,853 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 372 msec 2023-02-13 00:55:02,989 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:02,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,990 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:02,990 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,991 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:02,991 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:02,991 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,991 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,992 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:02,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:02,993 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:02,993 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:02,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,994 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,994 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:02,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:02,999 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:02,999 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:02,999 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,000 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,000 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,000 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:55:03,000 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,000 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,001 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,002 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,002 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,012 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/7e26c16101d94a419b992bcdae686ca3 2023-02-13 00:55:03,026 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/f659ae81b64c45a3b455b0b3bb310265 2023-02-13 00:55:03,030 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/7e26c16101d94a419b992bcdae686ca3 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3 2023-02-13 00:55:03,034 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:55:03,035 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/f659ae81b64c45a3b455b0b3bb310265 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265 2023-02-13 00:55:03,039 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:55:03,040 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 40ms, sequenceid=6, compaction requested=false 2023-02-13 00:55:03,040 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,040 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,040 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,040 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,040 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,040 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,040 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,040 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,043 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,043 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,043 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,043 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29b989d1[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,045 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,045 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,046 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,046 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,047 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,049 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,049 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,050 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,050 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,050 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,053 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,053 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,053 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,061 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,060 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,061 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,062 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,063 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,063 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,064 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,070 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,070 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,069 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,070 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,071 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,071 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,071 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,071 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,071 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,170 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,184 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,185 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,186 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,186 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:03,186 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:03,186 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,186 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,187 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,187 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,187 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,187 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,187 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,187 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,189 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,190 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,190 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,190 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,190 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:03,190 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,191 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,199 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/9266c40d9a3745069cae6c5581a8d0a4 2023-02-13 00:55:03,211 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 2023-02-13 00:55:03,215 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/9266c40d9a3745069cae6c5581a8d0a4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4 2023-02-13 00:55:03,220 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:55:03,220 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 2023-02-13 00:55:03,224 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:55:03,225 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 35ms, sequenceid=19, compaction requested=false 2023-02-13 00:55:03,225 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,225 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,225 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,225 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,225 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,225 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,225 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,228 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,228 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,228 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,228 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7726f8b[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,230 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,230 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,232 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,232 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,232 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,233 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,232 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,232 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,233 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,233 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,233 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,234 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,234 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,235 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,235 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,236 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,239 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,239 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,239 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,239 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,239 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,247 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,247 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,248 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,248 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,250 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,250 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,250 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,253 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,253 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,253 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,254 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,255 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,354 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,368 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,370 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,370 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,370 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:03,370 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:03,371 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,371 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,372 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,372 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,372 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,372 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,372 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,373 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,373 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,373 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,373 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,374 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,374 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,374 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,374 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:03,374 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:55:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,377 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,377 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,377 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,383 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/1e28643b9ebf48489016f33cd4773874 2023-02-13 00:55:03,395 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/6807517779484a6f861a062affc813e5 2023-02-13 00:55:03,399 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/1e28643b9ebf48489016f33cd4773874 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874 2023-02-13 00:55:03,403 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:55:03,404 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/6807517779484a6f861a062affc813e5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5 2023-02-13 00:55:03,408 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:55:03,409 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 35ms, sequenceid=32, compaction requested=true 2023-02-13 00:55:03,409 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,410 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,410 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,414 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,414 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,414 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,414 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,416 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42b7a27[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,416 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,416 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,417 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,417 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,417 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,418 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,418 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,419 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,419 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,421 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,421 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,422 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,421 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,422 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,422 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,426 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,426 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,426 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,426 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,426 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,428 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,429 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,430 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,430 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,432 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,432 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,433 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,436 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,436 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,436 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:03,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,437 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,537 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,551 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,553 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:03,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:03,554 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,554 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,555 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,555 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,556 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,556 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,557 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,557 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,558 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,557 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,558 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,558 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,558 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,560 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,561 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,561 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,561 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,579 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/8d374dca8cea4e92bf4872dff1063087 2023-02-13 00:55:03,591 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 2023-02-13 00:55:03,596 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/8d374dca8cea4e92bf4872dff1063087 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087 2023-02-13 00:55:03,600 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:55:03,601 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 2023-02-13 00:55:03,605 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:55:03,605 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 47ms, sequenceid=45, compaction requested=true 2023-02-13 00:55:03,606 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,606 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,606 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,606 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,606 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,606 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,606 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,607 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,608 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,608 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@33897d90[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,610 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,610 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,612 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,612 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,612 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,614 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,614 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,614 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,615 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,615 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,619 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,619 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,619 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,619 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,619 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,625 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,625 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,627 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,627 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,628 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,630 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,630 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,630 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,630 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,631 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,632 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,632 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,632 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,632 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,731 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,746 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,746 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,746 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:03,746 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:03,746 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,746 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,747 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,747 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,748 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,749 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,749 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,750 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:03,750 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:55:03,751 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,753 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,753 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,753 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,762 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 2023-02-13 00:55:03,772 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/c1d413bee8f141b1bdf89f5859466f02 2023-02-13 00:55:03,776 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 2023-02-13 00:55:03,780 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:55:03,781 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/c1d413bee8f141b1bdf89f5859466f02 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02 2023-02-13 00:55:03,785 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:55:03,786 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 36ms, sequenceid=58, compaction requested=true 2023-02-13 00:55:03,786 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,786 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,786 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,786 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,786 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,786 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,786 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,788 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,788 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,788 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@561b8d31[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,790 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,790 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,791 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,791 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,791 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,793 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,793 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,795 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,795 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,795 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,798 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,798 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,798 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,798 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,798 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,799 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,799 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,800 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,800 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,801 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,803 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,803 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,803 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,806 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,807 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,807 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,907 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:03,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:03,919 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:03,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:03,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,921 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:03,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:03,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:03,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,931 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:03,931 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,932 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:03,933 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:03,934 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:03,934 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,934 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:03,934 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:55:03,935 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,936 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,936 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,936 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,936 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,936 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,947 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/fa3ce77601a14357b2c52a5772b1533b 2023-02-13 00:55:03,970 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/d6700d7316ca464c932093826d10279c 2023-02-13 00:55:03,975 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/fa3ce77601a14357b2c52a5772b1533b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b 2023-02-13 00:55:03,979 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:55:03,979 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/d6700d7316ca464c932093826d10279c as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c 2023-02-13 00:55:03,983 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:55:03,984 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 50ms, sequenceid=71, compaction requested=true 2023-02-13 00:55:03,984 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:03,984 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:03,984 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:03,985 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,985 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,986 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,986 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:03,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4281be4e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:03,988 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:03,988 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,990 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,991 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:03,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:03,991 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,992 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,993 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,993 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:03,993 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,993 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:03,996 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:03,996 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:03,996 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:03,996 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:03,996 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,996 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,998 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:03,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,999 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:03,999 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:03,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,002 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,002 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,002 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,006 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,006 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,007 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,007 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,007 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,007 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,007 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,007 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,008 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,008 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,008 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,008 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,008 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,107 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:04,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:04,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:04,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:04,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,121 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,121 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,121 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:04,122 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:04,122 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,122 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,123 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,123 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,123 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,123 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,124 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,125 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,126 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:04,126 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:04,126 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,126 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:04,126 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,127 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,135 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,138 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,146 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,149 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,150 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/abfd99672ab24af8895e4efaf5175c6a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,153 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,153 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 00:55:04,154 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/b37ba8462d9d412da0c42322b23d34ba as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,157 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,157 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 00:55:04,158 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 32ms, sequenceid=83, compaction requested=true 2023-02-13 00:55:04,158 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:04,158 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,159 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,159 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,160 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,160 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:04,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1da41e7b[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:04,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:04,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,164 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:04,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:04,164 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,165 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,166 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,169 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,169 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,169 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,169 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,169 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,169 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,170 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:04,172 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:04,172 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:04,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:04,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:04,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,172 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,179 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,179 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,179 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,181 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,181 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,181 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,184 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,183 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,184 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,184 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:04,185 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,184 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,285 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:04,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:04,293 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:04,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:04,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,295 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,295 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,295 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:04,295 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:04,295 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,295 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,296 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,296 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,296 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,296 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,297 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,297 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,298 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:04,299 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:04,299 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,299 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:04,299 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,300 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,308 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,312 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,319 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,322 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,322 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/19a36d16736c427ebb119bf0d8b092f7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,325 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,325 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 00:55:04,326 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/e43a4a585063402caf5de8717752823b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,329 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,329 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 00:55:04,330 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 31ms, sequenceid=91, compaction requested=true 2023-02-13 00:55:04,330 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:04,330 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:04,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:04,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:04,335 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@65b985a2[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:04,335 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:04,335 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,336 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,337 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,338 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,339 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,339 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:04,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:04,341 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:04,341 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:04,341 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:04,341 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,341 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,344 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,345 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,346 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,347 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,347 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,352 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,352 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,353 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,453 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:04,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:04,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:55:04,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:55:04,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,462 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 00:55:04,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 00:55:04,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,464 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,465 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,465 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,465 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:55:04,466 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,466 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. started... 2023-02-13 00:55:04,467 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 00:55:04,468 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,469 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,475 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,478 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,485 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,489 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,489 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/44508811ed6d4c0190982a28dd74d435 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,494 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,494 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 00:55:04,494 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/ceba4dcd6a684016b10c1710ff62e75f as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,499 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,499 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 00:55:04,500 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 33ms, sequenceid=99, compaction requested=true 2023-02-13 00:55:04,500 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ae67fb354b1c30a4bea3ba0a8e540f12: 2023-02-13 00:55:04,500 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,500 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:55:04,500 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:55:04,500 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,500 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 00:55:04,500 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 00:55:04,501 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,502 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,502 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:55:04,502 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 00:55:04,504 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f41fe21[Count = 0] remaining members to acquire global barrier 2023-02-13 00:55:04,504 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 00:55:04,505 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:55:04,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 00:55:04,506 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,512 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,513 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,513 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 00:55:04,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:55:04,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:55:04,515 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 00:55:04,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:55:04,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:55:04,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,515 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:55:04,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,517 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,518 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:55:04,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,518 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,518 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:55:04,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,519 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:55:04,520 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,520 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:55:04,522 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,523 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:55:04,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 00:55:04,523 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:55:04,524 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,524 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:04,523 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:55:04,524 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:55:04,624 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:55:04,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:55:04,627 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,628 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:04,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:04,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:04,629 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:04,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, UNASSIGN}] 2023-02-13 00:55:04,632 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, UNASSIGN 2023-02-13 00:55:04,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ae67fb354b1c30a4bea3ba0a8e540f12, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:04,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249704632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249704632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249704632"}]},"ts":"1676249704632"} 2023-02-13 00:55:04,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure ae67fb354b1c30a4bea3ba0a8e540f12, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:04,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ae67fb354b1c30a4bea3ba0a8e540f12, disabling compactions & flushes 2023-02-13 00:55:04,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. after waiting 0 ms 2023-02-13 00:55:04,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing ae67fb354b1c30a4bea3ba0a8e540f12 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 00:55:04,795 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,799 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,806 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf1/163ae321b4404eeb86b693a83e994dc5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 00:55:04,814 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.tmp/cf2/1a0e500c23cd43319802f07854120783 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 00:55:04,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 32ms, sequenceid=103, compaction requested=true 2023-02-13 00:55:04,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 00:55:04,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:04,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. 2023-02-13 00:55:04,823 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ae67fb354b1c30a4bea3ba0a8e540f12: Waiting for close lock at 1676249704786Running coprocessor pre-close hooks at 1676249704786Disabling compacts and flushes for region at 1676249704786Disabling writes for close at 1676249704786Obtaining lock to block concurrent updates at 1676249704786Preparing flush snapshotting stores in ae67fb354b1c30a4bea3ba0a8e540f12 at 1676249704786Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676249704786Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12. at 1676249704787 (+1 ms)Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf1: creating writer at 1676249704787Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf1: appending metadata at 1676249704789 (+2 ms)Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf1: closing flushed file at 1676249704789Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf2: creating writer at 1676249704799 (+10 ms)Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf2: appending metadata at 1676249704800 (+1 ms)Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf2: closing flushed file at 1676249704800Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf1: reopening flushed file at 1676249704810 (+10 ms)Flushing ae67fb354b1c30a4bea3ba0a8e540f12/cf2: reopening flushed file at 1676249704814 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ae67fb354b1c30a4bea3ba0a8e540f12 in 32ms, sequenceid=103, compaction requested=true at 1676249704818 (+4 ms)Writing region close event to WAL at 1676249704820 (+2 ms)Running coprocessor post-close hooks at 1676249704823 (+3 ms)Closed at 1676249704823 2023-02-13 00:55:04,824 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ae67fb354b1c30a4bea3ba0a8e540f12, regionState=CLOSED 2023-02-13 00:55:04,825 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249704825"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249704825"}]},"ts":"1676249704825"} 2023-02-13 00:55:04,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 00:55:04,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure ae67fb354b1c30a4bea3ba0a8e540f12, server=jenkins-hbase4.apache.org,41539,1676249678442 in 192 msec 2023-02-13 00:55:04,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 00:55:04,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae67fb354b1c30a4bea3ba0a8e540f12, UNASSIGN in 196 msec 2023-02-13 00:55:04,842 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=ae67fb354b1c30a4bea3ba0a8e540f12, threads=16 2023-02-13 00:55:04,842 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,842 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,842 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,842 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,842 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,844 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,845 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,846 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,846 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,846 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,848 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,848 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,848 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,850 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,854 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,854 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,854 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,857 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,858 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899, top=false 2023-02-13 00:55:04,858 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c, top=false 2023-02-13 00:55:04,866 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,867 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,871 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02, top=false 2023-02-13 00:55:04,871 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:04,871 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f, top=true 2023-02-13 00:55:04,872 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,873 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087, top=false 2023-02-13 00:55:04,873 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,873 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,873 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a0e500c23cd43319802f07854120783 2023-02-13 00:55:04,874 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783, top=true 2023-02-13 00:55:04,876 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899 for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,876 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:04,876 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7, top=true 2023-02-13 00:55:04,877 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:04,877 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435, top=true 2023-02-13 00:55:04,888 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,888 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,889 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e43a4a585063402caf5de8717752823b 2023-02-13 00:55:04,890 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b, top=true 2023-02-13 00:55:04,890 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:04,890 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5, top=true 2023-02-13 00:55:04,893 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5 for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783 for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,895 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,897 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,897 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,897 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,900 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,901 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8, top=false 2023-02-13 00:55:04,903 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,903 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b, top=false 2023-02-13 00:55:04,908 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7 for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,908 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,908 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,908 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,912 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435 for child: e5a290b2c8b29817f0f0a685a173099d, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,912 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,919 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087 for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,922 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02 for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,927 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,950 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,955 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,956 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/.splits/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8 for child: 2031b155b1006fc9dadf482f4464cd65, parent: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,960 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,960 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:04,972 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,974 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,986 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:04,986 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:04,992 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,000 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,003 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,006 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,007 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4 for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,007 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a for region: ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,007 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region ae67fb354b1c30a4bea3ba0a8e540f12 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:55:05,053 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 00:55:05,055 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 00:55:05,057 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676249705057"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676249705057"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676249705057"}]},"ts":"1676249705057"} 2023-02-13 00:55:05,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705057"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249705057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249705057"}]},"ts":"1676249705057"} 2023-02-13 00:55:05,058 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705057"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249705057"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249705057"}]},"ts":"1676249705057"} 2023-02-13 00:55:05,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, ASSIGN}] 2023-02-13 00:55:05,062 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, ASSIGN 2023-02-13 00:55:05,063 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:55:05,063 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, ASSIGN 2023-02-13 00:55:05,064 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:55:05,213 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:55:05,215 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=2031b155b1006fc9dadf482f4464cd65, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705215"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249705215"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249705215"}]},"ts":"1676249705215"} 2023-02-13 00:55:05,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 2031b155b1006fc9dadf482f4464cd65, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:05,219 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e5a290b2c8b29817f0f0a685a173099d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,219 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705219"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249705219"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249705219"}]},"ts":"1676249705219"} 2023-02-13 00:55:05,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure e5a290b2c8b29817f0f0a685a173099d, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:05,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e5a290b2c8b29817f0f0a685a173099d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 00:55:05,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:05,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:05,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,371 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,372 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1 2023-02-13 00:55:05,372 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1 2023-02-13 00:55:05,373 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5a290b2c8b29817f0f0a685a173099d columnFamilyName cf1 2023-02-13 00:55:05,380 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:05,383 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-top 2023-02-13 00:55:05,387 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-top 2023-02-13 00:55:05,390 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-top 2023-02-13 00:55:05,393 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,395 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-top 2023-02-13 00:55:05,398 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:05,398 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:05,402 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:05,402 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:05,405 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:05,406 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:05,406 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(310): Store=e5a290b2c8b29817f0f0a685a173099d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:05,406 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,407 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2 2023-02-13 00:55:05,407 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2 2023-02-13 00:55:05,407 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5a290b2c8b29817f0f0a685a173099d columnFamilyName cf2 2023-02-13 00:55:05,419 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-top 2023-02-13 00:55:05,423 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-top 2023-02-13 00:55:05,426 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,427 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-top 2023-02-13 00:55:05,431 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-top 2023-02-13 00:55:05,433 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a0e500c23cd43319802f07854120783 2023-02-13 00:55:05,433 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783 2023-02-13 00:55:05,436 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:05,436 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:05,439 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e43a4a585063402caf5de8717752823b 2023-02-13 00:55:05,439 DEBUG [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b 2023-02-13 00:55:05,439 INFO [StoreOpener-e5a290b2c8b29817f0f0a685a173099d-1] regionserver.HStore(310): Store=e5a290b2c8b29817f0f0a685a173099d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:05,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,442 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:05,443 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e5a290b2c8b29817f0f0a685a173099d; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60648683, jitterRate=-0.09626419842243195}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:05,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e5a290b2c8b29817f0f0a685a173099d: Running coprocessor pre-open hook at 1676249705370Writing region info on filesystem at 1676249705370Initializing all the Stores at 1676249705371 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249705371Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249705371Cleaning up temporary data from old regions at 1676249705441 (+70 ms)Cleaning up detritus from prior splits at 1676249705441Running coprocessor post-open hooks at 1676249705444 (+3 ms)Region opened successfully at 1676249705444 2023-02-13 00:55:05,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d., pid=78, masterSystemTime=1676249705367 2023-02-13 00:55:05,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,446 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:55:05,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:05,446 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:55:05,446 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,446 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): e5a290b2c8b29817f0f0a685a173099d/cf1 is initiating minor compaction (all files) 2023-02-13 00:55:05,446 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e5a290b2c8b29817f0f0a685a173099d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,446 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,446 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): e5a290b2c8b29817f0f0a685a173099d/cf2 is initiating minor compaction (all files) 2023-02-13 00:55:05,446 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e5a290b2c8b29817f0f0a685a173099d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,446 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp, totalSize=34.8 K 2023-02-13 00:55:05,447 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp, totalSize=34.8 K 2023-02-13 00:55:05,447 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249702984 2023-02-13 00:55:05,447 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249703172 2023-02-13 00:55:05,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,448 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249702984 2023-02-13 00:55:05,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2031b155b1006fc9dadf482f4464cd65, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 00:55:05,448 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e5a290b2c8b29817f0f0a685a173099d, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,448 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249703356 2023-02-13 00:55:05,448 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705448"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249705448"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249705448"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249705448"}]},"ts":"1676249705448"} 2023-02-13 00:55:05,448 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249703172 2023-02-13 00:55:05,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:05,449 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676249704109 2023-02-13 00:55:05,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:05,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,449 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249703356 2023-02-13 00:55:05,449 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,450 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,450 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676249704109 2023-02-13 00:55:05,450 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,450 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,450 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,451 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,452 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,452 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1 2023-02-13 00:55:05,452 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1 2023-02-13 00:55:05,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-13 00:55:05,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure e5a290b2c8b29817f0f0a685a173099d, server=jenkins-hbase4.apache.org,41539,1676249678442 in 230 msec 2023-02-13 00:55:05,453 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2031b155b1006fc9dadf482f4464cd65 columnFamilyName cf1 2023-02-13 00:55:05,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, ASSIGN in 392 msec 2023-02-13 00:55:05,470 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-bottom 2023-02-13 00:55:05,470 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): e5a290b2c8b29817f0f0a685a173099d#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,476 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-bottom 2023-02-13 00:55:05,480 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-bottom 2023-02-13 00:55:05,488 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,488 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): e5a290b2c8b29817f0f0a685a173099d#cf2#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,488 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-bottom 2023-02-13 00:55:05,492 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087 2023-02-13 00:55:05,494 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8 2023-02-13 00:55:05,497 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b 2023-02-13 00:55:05,507 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8d3ea9b8b7a44b39bef80729b787bd73: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,508 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp/cf1/8d3ea9b8b7a44b39bef80729b787bd73 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/8d3ea9b8b7a44b39bef80729b787bd73 2023-02-13 00:55:05,508 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(310): Store=2031b155b1006fc9dadf482f4464cd65/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:05,508 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,510 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2 2023-02-13 00:55:05,510 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2 2023-02-13 00:55:05,510 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2031b155b1006fc9dadf482f4464cd65 columnFamilyName cf2 2023-02-13 00:55:05,513 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8d3ea9b8b7a44b39bef80729b787bd73: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,514 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e5a290b2c8b29817f0f0a685a173099d/cf1 of e5a290b2c8b29817f0f0a685a173099d into 8d3ea9b8b7a44b39bef80729b787bd73(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,514 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5a290b2c8b29817f0f0a685a173099d: 2023-02-13 00:55:05,514 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d., storeName=e5a290b2c8b29817f0f0a685a173099d/cf1, priority=9, startTime=1676249705445; duration=0sec 2023-02-13 00:55:05,514 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,522 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8b1ab056c11b41a2b354d0791151918b: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,522 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-bottom 2023-02-13 00:55:05,522 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp/cf2/8b1ab056c11b41a2b354d0791151918b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/8b1ab056c11b41a2b354d0791151918b 2023-02-13 00:55:05,525 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-bottom 2023-02-13 00:55:05,526 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 8b1ab056c11b41a2b354d0791151918b: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,527 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e5a290b2c8b29817f0f0a685a173099d/cf2 of e5a290b2c8b29817f0f0a685a173099d into 8b1ab056c11b41a2b354d0791151918b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,527 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5a290b2c8b29817f0f0a685a173099d: 2023-02-13 00:55:05,527 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d., storeName=e5a290b2c8b29817f0f0a685a173099d/cf2, priority=9, startTime=1676249705446; duration=0sec 2023-02-13 00:55:05,527 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,528 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,528 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-bottom 2023-02-13 00:55:05,531 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-bottom 2023-02-13 00:55:05,534 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899 2023-02-13 00:55:05,537 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02 2023-02-13 00:55:05,540 DEBUG [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c 2023-02-13 00:55:05,540 INFO [StoreOpener-2031b155b1006fc9dadf482f4464cd65-1] regionserver.HStore(310): Store=2031b155b1006fc9dadf482f4464cd65/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:05,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:55:05,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2031b155b1006fc9dadf482f4464cd65; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66794920, jitterRate=-0.004678130149841309}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:55:05,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2031b155b1006fc9dadf482f4464cd65: Running coprocessor pre-open hook at 1676249705449Writing region info on filesystem at 1676249705449Initializing all the Stores at 1676249705450 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249705450Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249705450Cleaning up temporary data from old regions at 1676249705542 (+92 ms)Cleaning up detritus from prior splits at 1676249705542Running coprocessor post-open hooks at 1676249705544 (+2 ms)Region opened successfully at 1676249705544 2023-02-13 00:55:05,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65., pid=77, masterSystemTime=1676249705367 2023-02-13 00:55:05,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:05,546 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:55:05,546 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:55:05,546 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,546 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): 2031b155b1006fc9dadf482f4464cd65/cf1 is initiating minor compaction (all files) 2023-02-13 00:55:05,546 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,546 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): 2031b155b1006fc9dadf482f4464cd65/cf2 is initiating minor compaction (all files) 2023-02-13 00:55:05,546 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2031b155b1006fc9dadf482f4464cd65/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,546 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2031b155b1006fc9dadf482f4464cd65/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,547 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp, totalSize=35.0 K 2023-02-13 00:55:05,547 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp, totalSize=35.0 K 2023-02-13 00:55:05,547 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249702984 2023-02-13 00:55:05,547 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249702984 2023-02-13 00:55:05,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,548 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249703172 2023-02-13 00:55:05,548 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249703172 2023-02-13 00:55:05,548 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249703356 2023-02-13 00:55:05,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=2031b155b1006fc9dadf482f4464cd65, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,548 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249703356 2023-02-13 00:55:05,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705548"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249705548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249705548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249705548"}]},"ts":"1676249705548"} 2023-02-13 00:55:05,548 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249703539 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249703539 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249703733 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249703733 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249703908 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249703908 2023-02-13 00:55:05,549 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249704109 2023-02-13 00:55:05,550 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249704109 2023-02-13 00:55:05,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-13 00:55:05,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 2031b155b1006fc9dadf482f4464cd65, server=jenkins-hbase4.apache.org,41539,1676249678442 in 333 msec 2023-02-13 00:55:05,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-13 00:55:05,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, ASSIGN in 491 msec 2023-02-13 00:55:05,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d in 927 msec 2023-02-13 00:55:05,564 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): 2031b155b1006fc9dadf482f4464cd65#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,564 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2031b155b1006fc9dadf482f4464cd65#cf1#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,575 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp/cf1/d6fd08c73e40427a9a802aeba8ba27fd as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/d6fd08c73e40427a9a802aeba8ba27fd 2023-02-13 00:55:05,576 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp/cf2/1d8f6d700ea248269d349217960b8e7e as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/1d8f6d700ea248269d349217960b8e7e 2023-02-13 00:55:05,585 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2031b155b1006fc9dadf482f4464cd65/cf2 of 2031b155b1006fc9dadf482f4464cd65 into 1d8f6d700ea248269d349217960b8e7e(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,585 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2031b155b1006fc9dadf482f4464cd65: 2023-02-13 00:55:05,586 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65., storeName=2031b155b1006fc9dadf482f4464cd65/cf2, priority=9, startTime=1676249705546; duration=0sec 2023-02-13 00:55:05,586 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,586 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2031b155b1006fc9dadf482f4464cd65/cf1 of 2031b155b1006fc9dadf482f4464cd65 into d6fd08c73e40427a9a802aeba8ba27fd(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,586 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2031b155b1006fc9dadf482f4464cd65: 2023-02-13 00:55:05,586 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65., storeName=2031b155b1006fc9dadf482f4464cd65/cf1, priority=9, startTime=1676249705545; duration=0sec 2023-02-13 00:55:05,586 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,639 DEBUG [Listener at localhost/43763] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 00:55:05,642 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36254, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 00:55:05,642 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HStore(1912): 2031b155b1006fc9dadf482f4464cd65/cf1 is initiating major compaction (all files) 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@689bbc06; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,644 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 2031b155b1006fc9dadf482f4464cd65/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,644 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/d6fd08c73e40427a9a802aeba8ba27fd] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp, totalSize=6.8 K 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HStore(1912): 2031b155b1006fc9dadf482f4464cd65/cf2 is initiating major compaction (all files) 2023-02-13 00:55:05,644 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6f6e92a2; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:05,644 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 2031b155b1006fc9dadf482f4464cd65/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,645 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/1d8f6d700ea248269d349217960b8e7e] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp, totalSize=6.8 K 2023-02-13 00:55:05,645 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting d6fd08c73e40427a9a802aeba8ba27fd, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249702984 2023-02-13 00:55:05,645 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 1d8f6d700ea248269d349217960b8e7e, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249702984 2023-02-13 00:55:05,646 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,646 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 00:55:05,646 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.HStore(1912): e5a290b2c8b29817f0f0a685a173099d/cf1 is initiating major compaction (all files) 2023-02-13 00:55:05,647 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@554b46e0; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:05,647 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 00:55:05,647 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.HStore(1912): e5a290b2c8b29817f0f0a685a173099d/cf2 is initiating major compaction (all files) 2023-02-13 00:55:05,647 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@114f5428; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 00:55:05,647 INFO [Listener at localhost/43763] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 00:55:05,653 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2031b155b1006fc9dadf482f4464cd65#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,654 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): 2031b155b1006fc9dadf482f4464cd65#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,659 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. is not mergeable because it has references 2023-02-13 00:55:05,660 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. is not mergeable because it has references 2023-02-13 00:55:05,673 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp/cf1/c1697a422d8a41b0852d2e81dce259fb as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/c1697a422d8a41b0852d2e81dce259fb 2023-02-13 00:55:05,675 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/.tmp/cf2/fc8e773d21e54ba5a51bfa5a3b6d9dfc as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/fc8e773d21e54ba5a51bfa5a3b6d9dfc 2023-02-13 00:55:05,677 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2031b155b1006fc9dadf482f4464cd65/cf1 of 2031b155b1006fc9dadf482f4464cd65 into c1697a422d8a41b0852d2e81dce259fb(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,677 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2031b155b1006fc9dadf482f4464cd65: 2023-02-13 00:55:05,678 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676249705644; duration=0sec 2023-02-13 00:55:05,678 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 00:55:05,678 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e5a290b2c8b29817f0f0a685a173099d/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,678 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/8d3ea9b8b7a44b39bef80729b787bd73] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp, totalSize=5.0 K 2023-02-13 00:55:05,679 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 2031b155b1006fc9dadf482f4464cd65/cf2 of 2031b155b1006fc9dadf482f4464cd65 into fc8e773d21e54ba5a51bfa5a3b6d9dfc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,679 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 2031b155b1006fc9dadf482f4464cd65: 2023-02-13 00:55:05,679 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676249705644; duration=0sec 2023-02-13 00:55:05,679 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting 8d3ea9b8b7a44b39bef80729b787bd73, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,679 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:55:05,679 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e5a290b2c8b29817f0f0a685a173099d/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,679 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/8b1ab056c11b41a2b354d0791151918b] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp, totalSize=5.0 K 2023-02-13 00:55:05,679 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting 8b1ab056c11b41a2b354d0791151918b, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 00:55:05,681 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): e5a290b2c8b29817f0f0a685a173099d#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,682 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): e5a290b2c8b29817f0f0a685a173099d#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:55:05,691 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2efefba191eb4cceb559a259d5dacc84: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,691 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 30d4e42606e4407bbadad78c1e63f351: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,691 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp/cf1/2efefba191eb4cceb559a259d5dacc84 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/2efefba191eb4cceb559a259d5dacc84 2023-02-13 00:55:05,692 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/.tmp/cf2/30d4e42606e4407bbadad78c1e63f351 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/30d4e42606e4407bbadad78c1e63f351 2023-02-13 00:55:05,695 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 2efefba191eb4cceb559a259d5dacc84: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,695 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 30d4e42606e4407bbadad78c1e63f351: NONE, but ROW specified in column family configuration 2023-02-13 00:55:05,696 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e5a290b2c8b29817f0f0a685a173099d/cf1 of e5a290b2c8b29817f0f0a685a173099d into 2efefba191eb4cceb559a259d5dacc84(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,696 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5a290b2c8b29817f0f0a685a173099d: 2023-02-13 00:55:05,696 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676249705646; duration=0sec 2023-02-13 00:55:05,696 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in e5a290b2c8b29817f0f0a685a173099d/cf2 of e5a290b2c8b29817f0f0a685a173099d into 30d4e42606e4407bbadad78c1e63f351(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:55:05,696 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,696 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e5a290b2c8b29817f0f0a685a173099d: 2023-02-13 00:55:05,696 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676249705647; duration=0sec 2023-02-13 00:55:05,696 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:55:05,767 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. is not mergeable because it has references 2023-02-13 00:55:05,768 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41539] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. is not mergeable because it has references 2023-02-13 00:55:05,773 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 00:55:05,775 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:05,776 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:05,776 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:05,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:05,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:05,779 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249705779"}]},"ts":"1676249705779"} 2023-02-13 00:55:05,780 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 00:55:05,783 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 00:55:05,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, UNASSIGN}] 2023-02-13 00:55:05,784 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, UNASSIGN 2023-02-13 00:55:05,784 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e5a290b2c8b29817f0f0a685a173099d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,785 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705784"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249705784"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249705784"}]},"ts":"1676249705784"} 2023-02-13 00:55:05,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure e5a290b2c8b29817f0f0a685a173099d, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:05,787 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, UNASSIGN 2023-02-13 00:55:05,788 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2031b155b1006fc9dadf482f4464cd65, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:05,788 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705788"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249705788"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249705788"}]},"ts":"1676249705788"} 2023-02-13 00:55:05,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 2031b155b1006fc9dadf482f4464cd65, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:05,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:05,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2031b155b1006fc9dadf482f4464cd65, disabling compactions & flushes 2023-02-13 00:55:05,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. after waiting 0 ms 2023-02-13 00:55:05,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,938 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/d6fd08c73e40427a9a802aeba8ba27fd, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-bottom] to archive 2023-02-13 00:55:05,939 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:05,940 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,941 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,942 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,943 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-8d374dca8cea4e92bf4872dff1063087 2023-02-13 00:55:05,944 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-9fc54e6f5f454d4f8dd5b7e7ab6044a8 2023-02-13 00:55:05,945 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-fa3ce77601a14357b2c52a5772b1533b 2023-02-13 00:55:05,946 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/d6fd08c73e40427a9a802aeba8ba27fd to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/d6fd08c73e40427a9a802aeba8ba27fd 2023-02-13 00:55:05,947 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,949 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/1d8f6d700ea248269d349217960b8e7e, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-bottom] to archive 2023-02-13 00:55:05,949 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:05,950 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,951 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,952 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,953 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-887ee0e8a7d64eb1b168ca83a3ef2899 2023-02-13 00:55:05,954 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-c1d413bee8f141b1bdf89f5859466f02 2023-02-13 00:55:05,956 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-d6700d7316ca464c932093826d10279c 2023-02-13 00:55:05,957 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/1d8f6d700ea248269d349217960b8e7e to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/1d8f6d700ea248269d349217960b8e7e 2023-02-13 00:55:05,958 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 00:55:05,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:05,962 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65. 2023-02-13 00:55:05,962 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2031b155b1006fc9dadf482f4464cd65: Waiting for close lock at 1676249705937Running coprocessor pre-close hooks at 1676249705937Disabling compacts and flushes for region at 1676249705937Disabling writes for close at 1676249705937Writing region close event to WAL at 1676249705959 (+22 ms)Running coprocessor post-close hooks at 1676249705961 (+2 ms)Closed at 1676249705962 (+1 ms) 2023-02-13 00:55:05,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:05,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e5a290b2c8b29817f0f0a685a173099d, disabling compactions & flushes 2023-02-13 00:55:05,963 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. after waiting 0 ms 2023-02-13 00:55:05,963 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,963 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2031b155b1006fc9dadf482f4464cd65, regionState=CLOSED 2023-02-13 00:55:05,963 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705963"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249705963"}]},"ts":"1676249705963"} 2023-02-13 00:55:05,964 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/8d3ea9b8b7a44b39bef80729b787bd73, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5] to archive 2023-02-13 00:55:05,965 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:05,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 00:55:05,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 2031b155b1006fc9dadf482f4464cd65, server=jenkins-hbase4.apache.org,41539,1676249678442 in 176 msec 2023-02-13 00:55:05,966 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/7e26c16101d94a419b992bcdae686ca3.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=2031b155b1006fc9dadf482f4464cd65, UNASSIGN in 184 msec 2023-02-13 00:55:05,968 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/9266c40d9a3745069cae6c5581a8d0a4.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,969 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/1e28643b9ebf48489016f33cd4773874.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,970 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/abfd99672ab24af8895e4efaf5175c6a.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,971 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:05,971 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:05,972 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/8d3ea9b8b7a44b39bef80729b787bd73 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/8d3ea9b8b7a44b39bef80729b787bd73 2023-02-13 00:55:05,973 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:05,975 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/8b1ab056c11b41a2b354d0791151918b, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783] to archive 2023-02-13 00:55:05,975 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:55:05,976 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/f659ae81b64c45a3b455b0b3bb310265.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,977 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/9ab18cc6ad7a4ac795127331ed5f6d31.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,978 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/6807517779484a6f861a062affc813e5.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,979 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/b37ba8462d9d412da0c42322b23d34ba.ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:05,980 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-e43a4a585063402caf5de8717752823b 2023-02-13 00:55:05,981 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:05,982 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/8b1ab056c11b41a2b354d0791151918b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/8b1ab056c11b41a2b354d0791151918b 2023-02-13 00:55:05,983 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/testSplitTableRegionDeletedRowsDaughter=ae67fb354b1c30a4bea3ba0a8e540f12-1a0e500c23cd43319802f07854120783 2023-02-13 00:55:05,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 00:55:05,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:05,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d. 2023-02-13 00:55:05,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e5a290b2c8b29817f0f0a685a173099d: Waiting for close lock at 1676249705963Running coprocessor pre-close hooks at 1676249705963Disabling compacts and flushes for region at 1676249705963Disabling writes for close at 1676249705963Writing region close event to WAL at 1676249705983 (+20 ms)Running coprocessor post-close hooks at 1676249705986 (+3 ms)Closed at 1676249705987 (+1 ms) 2023-02-13 00:55:05,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:05,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e5a290b2c8b29817f0f0a685a173099d, regionState=CLOSED 2023-02-13 00:55:05,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676249705989"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249705989"}]},"ts":"1676249705989"} 2023-02-13 00:55:05,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 00:55:05,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure e5a290b2c8b29817f0f0a685a173099d, server=jenkins-hbase4.apache.org,41539,1676249678442 in 206 msec 2023-02-13 00:55:05,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-13 00:55:05,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e5a290b2c8b29817f0f0a685a173099d, UNASSIGN in 209 msec 2023-02-13 00:55:05,994 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249705994"}]},"ts":"1676249705994"} 2023-02-13 00:55:05,995 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 00:55:05,996 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 00:55:05,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 221 msec 2023-02-13 00:55:06,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 00:55:06,081 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 00:55:06,082 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,084 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,085 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 00:55:06,089 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:06,089 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:06,089 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:06,091 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/recovered.edits] 2023-02-13 00:55:06,091 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/recovered.edits] 2023-02-13 00:55:06,091 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/recovered.edits] 2023-02-13 00:55:06,094 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/2efefba191eb4cceb559a259d5dacc84 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf1/2efefba191eb4cceb559a259d5dacc84 2023-02-13 00:55:06,094 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/163ae321b4404eeb86b693a83e994dc5 2023-02-13 00:55:06,094 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/c1697a422d8a41b0852d2e81dce259fb to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf1/c1697a422d8a41b0852d2e81dce259fb 2023-02-13 00:55:06,095 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/19a36d16736c427ebb119bf0d8b092f7 2023-02-13 00:55:06,096 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/fc8e773d21e54ba5a51bfa5a3b6d9dfc to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/cf2/fc8e773d21e54ba5a51bfa5a3b6d9dfc 2023-02-13 00:55:06,096 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/30d4e42606e4407bbadad78c1e63f351 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/cf2/30d4e42606e4407bbadad78c1e63f351 2023-02-13 00:55:06,096 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/1e28643b9ebf48489016f33cd4773874 2023-02-13 00:55:06,097 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/44508811ed6d4c0190982a28dd74d435 2023-02-13 00:55:06,099 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/recovered.edits/113.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65/recovered.edits/113.seqid 2023-02-13 00:55:06,099 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/7e26c16101d94a419b992bcdae686ca3 2023-02-13 00:55:06,099 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/recovered.edits/113.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d/recovered.edits/113.seqid 2023-02-13 00:55:06,099 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/2031b155b1006fc9dadf482f4464cd65 2023-02-13 00:55:06,100 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:55:06,100 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/8d374dca8cea4e92bf4872dff1063087 2023-02-13 00:55:06,101 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9266c40d9a3745069cae6c5581a8d0a4 2023-02-13 00:55:06,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/9fc54e6f5f454d4f8dd5b7e7ab6044a8 2023-02-13 00:55:06,103 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/abfd99672ab24af8895e4efaf5175c6a 2023-02-13 00:55:06,104 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf1/fa3ce77601a14357b2c52a5772b1533b 2023-02-13 00:55:06,106 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/1a0e500c23cd43319802f07854120783 2023-02-13 00:55:06,107 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/6807517779484a6f861a062affc813e5 2023-02-13 00:55:06,109 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/887ee0e8a7d64eb1b168ca83a3ef2899 2023-02-13 00:55:06,110 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/9ab18cc6ad7a4ac795127331ed5f6d31 2023-02-13 00:55:06,111 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/b37ba8462d9d412da0c42322b23d34ba 2023-02-13 00:55:06,112 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/c1d413bee8f141b1bdf89f5859466f02 2023-02-13 00:55:06,113 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/ceba4dcd6a684016b10c1710ff62e75f 2023-02-13 00:55:06,114 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/d6700d7316ca464c932093826d10279c 2023-02-13 00:55:06,115 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/e43a4a585063402caf5de8717752823b 2023-02-13 00:55:06,116 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/cf2/f659ae81b64c45a3b455b0b3bb310265 2023-02-13 00:55:06,118 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/recovered.edits/106.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12/recovered.edits/106.seqid 2023-02-13 00:55:06,119 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae67fb354b1c30a4bea3ba0a8e540f12 2023-02-13 00:55:06,119 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 00:55:06,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,123 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 00:55:06,124 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 00:55:06,125 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,125 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 00:55:06,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249706125"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:06,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249706125"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:06,126 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249706125"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:06,127 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:55:06,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ae67fb354b1c30a4bea3ba0a8e540f12, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676249702480.ae67fb354b1c30a4bea3ba0a8e540f12.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2031b155b1006fc9dadf482f4464cd65, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676249704627.2031b155b1006fc9dadf482f4464cd65.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => e5a290b2c8b29817f0f0a685a173099d, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676249704627.e5a290b2c8b29817f0f0a685a173099d.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 00:55:06,127 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 00:55:06,127 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249706127"}]},"ts":"9223372036854775807"} 2023-02-13 00:55:06,128 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 00:55:06,130 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:55:06,131 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 48 msec 2023-02-13 00:55:06,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 00:55:06,186 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 00:55:06,200 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407 (was 403) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:35144 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:54748 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:35152 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:39240 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:56810 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:35202 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:55022 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:40858 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:41104 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:47668 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:55050 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:56542 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:57682 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:41126 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:35158 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1653417319_15 at /127.0.0.1:40052 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5adffcfe-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=863 (was 861) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=221 (was 223), ProcessCount=167 (was 167), AvailableMemoryMB=3480 (was 3492) 2023-02-13 00:55:06,214 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=407, OpenFileDescriptor=863, MaxFileDescriptor=60000, SystemLoadAverage=221, ProcessCount=167, AvailableMemoryMB=3479 2023-02-13 00:55:06,215 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:55:06,217 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:55:06,218 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:55:06,219 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:55:06,220 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:55:06,221 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:55:06,222 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,223 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 empty. 2023-02-13 00:55:06,223 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,223 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 00:55:06,232 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:06,233 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 27681efa2c518abe27ce49fb434b6528, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 27681efa2c518abe27ce49fb434b6528, disabling compactions & flushes 2023-02-13 00:55:06,244 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. after waiting 0 ms 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,244 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,244 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 27681efa2c518abe27ce49fb434b6528: Waiting for close lock at 1676249706244Disabling compacts and flushes for region at 1676249706244Disabling writes for close at 1676249706244Writing region close event to WAL at 1676249706244Closed at 1676249706244 2023-02-13 00:55:06,246 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:55:06,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249706246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249706246"}]},"ts":"1676249706246"} 2023-02-13 00:55:06,248 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:55:06,248 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:55:06,249 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249706248"}]},"ts":"1676249706248"} 2023-02-13 00:55:06,249 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 00:55:06,253 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:55:06,254 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:55:06,254 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:55:06,254 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:55:06,254 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:55:06,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, ASSIGN}] 2023-02-13 00:55:06,255 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, ASSIGN 2023-02-13 00:55:06,256 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:55:06,406 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:55:06,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:06,409 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249706409"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249706409"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249706409"}]},"ts":"1676249706409"} 2023-02-13 00:55:06,410 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:06,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 27681efa2c518abe27ce49fb434b6528, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:06,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:06,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:06,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,567 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,568 DEBUG [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:55:06,568 DEBUG [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:55:06,568 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27681efa2c518abe27ce49fb434b6528 columnFamilyName cf1 2023-02-13 00:55:06,569 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] regionserver.HStore(310): Store=27681efa2c518abe27ce49fb434b6528/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:06,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:55:06,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 27681efa2c518abe27ce49fb434b6528; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61887852, jitterRate=-0.07779914140701294}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:55:06,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 27681efa2c518abe27ce49fb434b6528: Running coprocessor pre-open hook at 1676249706565Writing region info on filesystem at 1676249706565Initializing all the Stores at 1676249706566 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249706566Cleaning up temporary data from old regions at 1676249706571 (+5 ms)Cleaning up detritus from prior splits at 1676249706571Running coprocessor post-open hooks at 1676249706580 (+9 ms)Region opened successfully at 1676249706580 2023-02-13 00:55:06,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528., pid=87, masterSystemTime=1676249706562 2023-02-13 00:55:06,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,583 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:06,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249706583"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249706583"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249706583"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249706583"}]},"ts":"1676249706583"} 2023-02-13 00:55:06,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 00:55:06,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414 in 175 msec 2023-02-13 00:55:06,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 00:55:06,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, ASSIGN in 333 msec 2023-02-13 00:55:06,590 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:55:06,590 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249706590"}]},"ts":"1676249706590"} 2023-02-13 00:55:06,591 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 00:55:06,596 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:55:06,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 378 msec 2023-02-13 00:55:06,728 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 00:55:06,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:55:06,747 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:06,748 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 00:55:06,748 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 00:55:06,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 00:55:06,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, REOPEN/MOVE}] 2023-02-13 00:55:06,763 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, REOPEN/MOVE 2023-02-13 00:55:06,764 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:06,764 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249706763"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249706763"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249706763"}]},"ts":"1676249706763"} 2023-02-13 00:55:06,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:06,916 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,917 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 27681efa2c518abe27ce49fb434b6528, disabling compactions & flushes 2023-02-13 00:55:06,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. after waiting 0 ms 2023-02-13 00:55:06,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 00:55:06,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:55:06,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:06,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 27681efa2c518abe27ce49fb434b6528: Waiting for close lock at 1676249706917Running coprocessor pre-close hooks at 1676249706917Disabling compacts and flushes for region at 1676249706917Disabling writes for close at 1676249706918 (+1 ms)Writing region close event to WAL at 1676249706918Running coprocessor post-close hooks at 1676249706921 (+3 ms)Closed at 1676249706921 2023-02-13 00:55:06,921 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 27681efa2c518abe27ce49fb434b6528 to self. 2023-02-13 00:55:06,922 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:06,923 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=CLOSED 2023-02-13 00:55:06,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249706923"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249706923"}]},"ts":"1676249706923"} 2023-02-13 00:55:06,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 00:55:06,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414 in 159 msec 2023-02-13 00:55:06,926 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=true 2023-02-13 00:55:07,076 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:55:07,077 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:07,077 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249707077"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249707077"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249707077"}]},"ts":"1676249707077"} 2023-02-13 00:55:07,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:55:07,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:07,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 27681efa2c518abe27ce49fb434b6528, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:55:07,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:07,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:07,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,234 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,235 DEBUG [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:55:07,235 DEBUG [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:55:07,235 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27681efa2c518abe27ce49fb434b6528 columnFamilyName cf1 2023-02-13 00:55:07,236 INFO [StoreOpener-27681efa2c518abe27ce49fb434b6528-1] regionserver.HStore(310): Store=27681efa2c518abe27ce49fb434b6528/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:55:07,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:55:07,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 27681efa2c518abe27ce49fb434b6528; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72180126, jitterRate=0.07556769251823425}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:55:07,240 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 27681efa2c518abe27ce49fb434b6528: Running coprocessor pre-open hook at 1676249707233Writing region info on filesystem at 1676249707233Initializing all the Stores at 1676249707234 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249707234Cleaning up temporary data from old regions at 1676249707237 (+3 ms)Cleaning up detritus from prior splits at 1676249707237Running coprocessor post-open hooks at 1676249707240 (+3 ms)Region opened successfully at 1676249707240 2023-02-13 00:55:07,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528., pid=92, masterSystemTime=1676249707230 2023-02-13 00:55:07,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:07,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:55:07,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:55:07,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249707242"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249707242"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249707242"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249707242"}]},"ts":"1676249707242"} 2023-02-13 00:55:07,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 00:55:07,245 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414 in 165 msec 2023-02-13 00:55:07,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 00:55:07,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, REOPEN/MOVE in 483 msec 2023-02-13 00:55:07,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 00:55:07,249 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 497 msec 2023-02-13 00:55:07,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, ASSIGN}] 2023-02-13 00:55:07,252 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, ASSIGN 2023-02-13 00:55:07,253 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 00:55:07,403 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:55:07,404 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:55:07,404 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:55:07,404 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:55:07,404 DEBUG [jenkins-hbase4:43619] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 00:55:07,405 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=dd5c9c772a71d6dc99be51911a2bcfb3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:55:07,406 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249707405"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676249707405"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676249707405"}]},"ts":"1676249707405"} 2023-02-13 00:55:07,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure dd5c9c772a71d6dc99be51911a2bcfb3, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:55:07,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:55:07,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dd5c9c772a71d6dc99be51911a2bcfb3, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 00:55:07,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:55:07,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:55:07,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:55:07,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:55:07,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:55:07,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:07,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:07,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:07,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:07,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:08,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,703 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 00:55:09,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:09,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,325 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:55:10,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:10,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:11,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:12,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:13,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:14,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:15,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:16,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:17,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:18,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:19,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:20,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:21,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:22,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:23,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:24,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:25,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:26,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:27,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:28,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:29,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:30,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:31,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:32,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:33,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:34,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:35,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:36,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:37,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:38,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,704 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:55:39,704 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:55:39,706 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 00:55:39,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:39,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:40,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:41,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:42,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:43,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:44,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:45,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:46,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:47,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:48,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:49,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:50,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:51,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:52,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:53,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:54,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:55,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:56,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:57,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:58,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:55:59,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:00,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:01,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:02,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:03,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:04,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:05,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:56:06,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:06,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:56:06,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:56:07,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:56:07,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:07,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 00:56:07,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:08,951 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:08,952 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:08,953 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:08,953 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:08,953 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:08,955 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 27681efa2c518abe27ce49fb434b6528, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:08,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 exec-time=6 msec 2023-02-13 00:56:08,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:09,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:09,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 00:56:09,283 INFO [StoreOpener-dd5c9c772a71d6dc99be51911a2bcfb3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:56:09,287 DEBUG [StoreOpener-dd5c9c772a71d6dc99be51911a2bcfb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:56:09,288 DEBUG [StoreOpener-dd5c9c772a71d6dc99be51911a2bcfb3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1 2023-02-13 00:56:09,288 INFO [StoreOpener-dd5c9c772a71d6dc99be51911a2bcfb3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd5c9c772a71d6dc99be51911a2bcfb3 columnFamilyName cf1 2023-02-13 00:56:09,289 INFO [StoreOpener-dd5c9c772a71d6dc99be51911a2bcfb3-1] regionserver.HStore(310): Store=dd5c9c772a71d6dc99be51911a2bcfb3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:09,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dd5c9c772a71d6dc99be51911a2bcfb3; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65469086, jitterRate=-0.024434596300125122}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 00:56:09,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dd5c9c772a71d6dc99be51911a2bcfb3: Running coprocessor pre-open hook at 1676249707564Initializing all the Stores at 1676249769283 (+61719 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249769283Running coprocessor post-open hooks at 1676249769291 (+8 ms)Region opened successfully at 1676249769291 2023-02-13 00:56:09,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3., pid=94, masterSystemTime=1676249707560 2023-02-13 00:56:09,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:09,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:09,293 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=dd5c9c772a71d6dc99be51911a2bcfb3, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:09,293 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249769293"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1676249769293"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676249769293"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676249769293"}]},"ts":"1676249769293"} 2023-02-13 00:56:09,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 00:56:09,296 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure dd5c9c772a71d6dc99be51911a2bcfb3, server=jenkins-hbase4.apache.org,41539,1676249678442 in 1 mins, 1.887 sec 2023-02-13 00:56:09,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 00:56:09,298 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, ASSIGN in 1 mins, 2.046 sec 2023-02-13 00:56:09,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.568 sec 2023-02-13 00:56:09,300 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676249706726","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62574,"client":"172.31.14.131:58626","queuetimems":0,"class":"HMaster"} 2023-02-13 00:56:09,301 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-13 00:56:11,203 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up 2023-02-13 00:56:11,205 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:11,206 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,206 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:56:11,209 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249771209"}]},"ts":"1676249771209"} 2023-02-13 00:56:11,210 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 00:56:11,212 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 00:56:11,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, UNASSIGN}] 2023-02-13 00:56:11,213 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, UNASSIGN 2023-02-13 00:56:11,214 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=dd5c9c772a71d6dc99be51911a2bcfb3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:11,214 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249771213"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1676249771213"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676249771213"}]},"ts":"1676249771213"} 2023-02-13 00:56:11,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure dd5c9c772a71d6dc99be51911a2bcfb3, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:11,216 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, UNASSIGN 2023-02-13 00:56:11,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:11,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249771217"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249771217"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249771217"}]},"ts":"1676249771217"} 2023-02-13 00:56:11,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:11,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:56:11,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:56:11,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dd5c9c772a71d6dc99be51911a2bcfb3, disabling compactions & flushes 2023-02-13 00:56:11,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:11,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:11,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. after waiting 0 ms 2023-02-13 00:56:11,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:11,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:11,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3. 2023-02-13 00:56:11,369 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dd5c9c772a71d6dc99be51911a2bcfb3: Waiting for close lock at 1676249771367Running coprocessor pre-close hooks at 1676249771367Disabling compacts and flushes for region at 1676249771367Disabling writes for close at 1676249771368 (+1 ms)Writing region close event to WAL at 1676249771368Running coprocessor post-close hooks at 1676249771368Closed at 1676249771369 (+1 ms) 2023-02-13 00:56:11,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:56:11,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 27681efa2c518abe27ce49fb434b6528, disabling compactions & flushes 2023-02-13 00:56:11,370 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:56:11,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:56:11,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. after waiting 0 ms 2023-02-13 00:56:11,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:56:11,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dd5c9c772a71d6dc99be51911a2bcfb3 2023-02-13 00:56:11,372 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=dd5c9c772a71d6dc99be51911a2bcfb3, regionState=CLOSED 2023-02-13 00:56:11,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249771371"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676249771371"}]},"ts":"1676249771371"} 2023-02-13 00:56:11,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 00:56:11,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 00:56:11,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure dd5c9c772a71d6dc99be51911a2bcfb3, server=jenkins-hbase4.apache.org,41539,1676249678442 in 159 msec 2023-02-13 00:56:11,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:11,377 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528. 2023-02-13 00:56:11,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 27681efa2c518abe27ce49fb434b6528: Waiting for close lock at 1676249771370Running coprocessor pre-close hooks at 1676249771370Disabling compacts and flushes for region at 1676249771370Disabling writes for close at 1676249771370Writing region close event to WAL at 1676249771371 (+1 ms)Running coprocessor post-close hooks at 1676249771376 (+5 ms)Closed at 1676249771377 (+1 ms) 2023-02-13 00:56:11,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3, UNASSIGN in 164 msec 2023-02-13 00:56:11,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 27681efa2c518abe27ce49fb434b6528 2023-02-13 00:56:11,378 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=27681efa2c518abe27ce49fb434b6528, regionState=CLOSED 2023-02-13 00:56:11,378 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676249771378"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249771378"}]},"ts":"1676249771378"} 2023-02-13 00:56:11,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 00:56:11,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 27681efa2c518abe27ce49fb434b6528, server=jenkins-hbase4.apache.org,34557,1676249678414 in 161 msec 2023-02-13 00:56:11,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 00:56:11,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=27681efa2c518abe27ce49fb434b6528, UNASSIGN in 169 msec 2023-02-13 00:56:11,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249771383"}]},"ts":"1676249771383"} 2023-02-13 00:56:11,384 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 00:56:11,385 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 00:56:11,387 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 180 msec 2023-02-13 00:56:11,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 00:56:11,511 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 00:56:11,512 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 00:56:11,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,519 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:56:11,520 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits] 2023-02-13 00:56:11,524 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits/7.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528/recovered.edits/7.seqid 2023-02-13 00:56:11,524 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/27681efa2c518abe27ce49fb434b6528 2023-02-13 00:56:11,525 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 00:56:11,527 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,528 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 00:56:11,530 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 00:56:11,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,531 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 00:56:11,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249771531"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:11,531 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249771531"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:11,532 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 00:56:11,532 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 27681efa2c518abe27ce49fb434b6528, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218.27681efa2c518abe27ce49fb434b6528.', STARTKEY => '', ENDKEY => ''}, {ENCODED => dd5c9c772a71d6dc99be51911a2bcfb3, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676249706218_0001.dd5c9c772a71d6dc99be51911a2bcfb3.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 00:56:11,532 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 00:56:11,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249771532"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:11,534 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 00:56:11,537 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:11,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 25 msec 2023-02-13 00:56:11,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 00:56:11,615 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 00:56:11,628 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=344 (was 407), OpenFileDescriptor=658 (was 863), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 221), ProcessCount=167 (was 167), AvailableMemoryMB=3302 (was 3479) 2023-02-13 00:56:11,639 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=345, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=167, AvailableMemoryMB=3301 2023-02-13 00:56:11,640 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:11,642 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:56:11,642 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:56:11,643 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:11,644 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:11,645 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:11,647 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,647 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d empty. 2023-02-13 00:56:11,647 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,647 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 00:56:11,656 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:11,657 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 02c4b8e12916f904a963dc699cdc889d, NAME => 'testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:56:11,663 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:11,663 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 02c4b8e12916f904a963dc699cdc889d, disabling compactions & flushes 2023-02-13 00:56:11,663 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,663 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,664 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. after waiting 0 ms 2023-02-13 00:56:11,664 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,664 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,664 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 02c4b8e12916f904a963dc699cdc889d: Waiting for close lock at 1676249771663Disabling compacts and flushes for region at 1676249771663Disabling writes for close at 1676249771664 (+1 ms)Writing region close event to WAL at 1676249771664Closed at 1676249771664 2023-02-13 00:56:11,665 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:11,666 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676249771666"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249771666"}]},"ts":"1676249771666"} 2023-02-13 00:56:11,667 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:56:11,668 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:11,668 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249771668"}]},"ts":"1676249771668"} 2023-02-13 00:56:11,669 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 00:56:11,672 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:56:11,672 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:56:11,672 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:56:11,672 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:56:11,672 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:56:11,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, ASSIGN}] 2023-02-13 00:56:11,674 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, ASSIGN 2023-02-13 00:56:11,674 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:56:11,824 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:56:11,825 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=02c4b8e12916f904a963dc699cdc889d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:11,826 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676249771825"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249771825"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249771825"}]},"ts":"1676249771825"} 2023-02-13 00:56:11,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 02c4b8e12916f904a963dc699cdc889d, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:11,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 02c4b8e12916f904a963dc699cdc889d, NAME => 'testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:56:11,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:11,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:11,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,982 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,983 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,984 DEBUG [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1 2023-02-13 00:56:11,984 DEBUG [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1 2023-02-13 00:56:11,985 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 02c4b8e12916f904a963dc699cdc889d columnFamilyName cf1 2023-02-13 00:56:11,985 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] regionserver.HStore(310): Store=02c4b8e12916f904a963dc699cdc889d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:11,985 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,986 DEBUG [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2 2023-02-13 00:56:11,986 DEBUG [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2 2023-02-13 00:56:11,986 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 02c4b8e12916f904a963dc699cdc889d columnFamilyName cf2 2023-02-13 00:56:11,987 INFO [StoreOpener-02c4b8e12916f904a963dc699cdc889d-1] regionserver.HStore(310): Store=02c4b8e12916f904a963dc699cdc889d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:11,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:11,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:11,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:11,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 02c4b8e12916f904a963dc699cdc889d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63106085, jitterRate=-0.05964605510234833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:11,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 02c4b8e12916f904a963dc699cdc889d: Running coprocessor pre-open hook at 1676249771982Writing region info on filesystem at 1676249771982Initializing all the Stores at 1676249771983 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249771983Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249771983Cleaning up temporary data from old regions at 1676249771988 (+5 ms)Cleaning up detritus from prior splits at 1676249771989 (+1 ms)Running coprocessor post-open hooks at 1676249771993 (+4 ms)Region opened successfully at 1676249771993 2023-02-13 00:56:11,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d., pid=104, masterSystemTime=1676249771979 2023-02-13 00:56:11,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:11,995 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=02c4b8e12916f904a963dc699cdc889d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:11,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676249771995"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249771995"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249771995"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249771995"}]},"ts":"1676249771995"} 2023-02-13 00:56:11,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 00:56:11,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 02c4b8e12916f904a963dc699cdc889d, server=jenkins-hbase4.apache.org,41539,1676249678442 in 170 msec 2023-02-13 00:56:12,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 00:56:12,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, ASSIGN in 326 msec 2023-02-13 00:56:12,001 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:12,001 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249772001"}]},"ts":"1676249772001"} 2023-02-13 00:56:12,002 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 00:56:12,005 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:12,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 362 msec 2023-02-13 00:56:12,151 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:12,153 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:12,153 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,153 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:12,153 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:12,153 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,154 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,155 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,155 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,155 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,157 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,157 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,157 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,157 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:12,158 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:56:12,159 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,160 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,160 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,160 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,160 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,160 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,172 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/e1a9261579b84a77a5fab3715549f186 2023-02-13 00:56:12,184 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/31ee78c549c34a49b209b84d45ee6f7e 2023-02-13 00:56:12,188 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/e1a9261579b84a77a5fab3715549f186 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/e1a9261579b84a77a5fab3715549f186 2023-02-13 00:56:12,192 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/e1a9261579b84a77a5fab3715549f186, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:12,193 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/31ee78c549c34a49b209b84d45ee6f7e as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/31ee78c549c34a49b209b84d45ee6f7e 2023-02-13 00:56:12,196 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/31ee78c549c34a49b209b84d45ee6f7e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:12,197 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 39ms, sequenceid=6, compaction requested=false 2023-02-13 00:56:12,197 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 00:56:12,198 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:12,198 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,198 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,198 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,198 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,198 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,198 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,200 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,200 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,200 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,201 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:12,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20fd5e49[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,202 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:12,203 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,203 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,203 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,203 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:12,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,204 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:12,205 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,205 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,205 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,205 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,206 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,206 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,206 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,208 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,208 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,208 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:12,210 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,210 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:12,210 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,210 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,210 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:12,210 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,212 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,212 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,213 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,213 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,215 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,215 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,215 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,218 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:12,219 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,218 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:12,218 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,219 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,319 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:12,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:12,330 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:12,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:12,332 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:12,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:12,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:12,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,333 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,334 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,334 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,335 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,335 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,335 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,335 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,335 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,336 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,336 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,336 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,336 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:12,336 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,341 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,346 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/0a8bea82e9004522ac4d9c92eb333e57 2023-02-13 00:56:12,358 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/9cbfa255be2346bf956c51a27ea2b4c2 2023-02-13 00:56:12,362 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/0a8bea82e9004522ac4d9c92eb333e57 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57 2023-02-13 00:56:12,365 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:12,366 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/9cbfa255be2346bf956c51a27ea2b4c2 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2 2023-02-13 00:56:12,370 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:12,371 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 35ms, sequenceid=19, compaction requested=false 2023-02-13 00:56:12,371 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:12,371 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,371 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,371 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,371 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,371 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,371 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,372 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,372 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,373 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,373 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:12,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d97cbeb[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,374 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:12,375 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,376 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,376 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:12,376 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,378 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,378 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:12,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,381 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:12,381 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,381 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,381 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:12,381 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,384 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,385 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,385 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,386 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,387 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,387 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,388 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,390 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:12,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,390 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,391 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,491 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:12,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:12,503 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:12,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:12,504 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:12,504 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,505 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:12,505 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:12,505 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,505 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,506 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,506 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,507 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,507 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,508 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,508 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,509 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,509 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,509 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,509 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,509 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:12,509 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:12,510 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,511 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,511 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,518 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/79f70e17e04e4ae2ab8d78df5456b63b 2023-02-13 00:56:12,529 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/2cf057010cf04a129249de544eeb62c8 2023-02-13 00:56:12,532 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/79f70e17e04e4ae2ab8d78df5456b63b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/79f70e17e04e4ae2ab8d78df5456b63b 2023-02-13 00:56:12,536 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/79f70e17e04e4ae2ab8d78df5456b63b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:12,537 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/2cf057010cf04a129249de544eeb62c8 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/2cf057010cf04a129249de544eeb62c8 2023-02-13 00:56:12,540 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/2cf057010cf04a129249de544eeb62c8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:12,541 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 32ms, sequenceid=32, compaction requested=true 2023-02-13 00:56:12,541 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:12,541 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,541 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,541 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,541 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,541 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,541 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,542 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,542 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,543 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,543 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:12,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34aac96e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,545 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:12,545 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,546 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,546 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:12,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,547 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,547 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,547 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,548 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,549 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,549 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,550 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,550 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:12,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,552 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:12,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,553 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:12,553 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,554 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,555 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,555 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,556 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,557 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,557 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,563 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:12,563 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,563 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,562 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:12,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,563 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,663 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:12,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:12,677 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:12,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:12,678 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:12,678 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,678 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:12,679 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:12,679 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,679 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,681 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,681 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,681 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,682 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,683 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,683 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,683 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,684 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,684 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,684 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:12,684 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:12,685 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,685 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,685 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,685 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,686 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,700 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/1f4f51cd63de405c89d982e64b8a559f 2023-02-13 00:56:12,710 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/dfb521ff9980475fabc259843802006d 2023-02-13 00:56:12,717 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/1f4f51cd63de405c89d982e64b8a559f as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/1f4f51cd63de405c89d982e64b8a559f 2023-02-13 00:56:12,731 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/1f4f51cd63de405c89d982e64b8a559f, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:12,732 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/dfb521ff9980475fabc259843802006d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/dfb521ff9980475fabc259843802006d 2023-02-13 00:56:12,735 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/dfb521ff9980475fabc259843802006d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:12,736 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 52ms, sequenceid=45, compaction requested=true 2023-02-13 00:56:12,736 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:12,736 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,738 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,738 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,738 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,738 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:12,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54fa992c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,740 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:12,741 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,742 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,743 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,743 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:12,743 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:12,744 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,744 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,744 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,745 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:12,750 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,750 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:12,751 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,751 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,751 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:12,751 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,752 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,752 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,752 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,752 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,753 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,753 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,754 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,756 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,758 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,758 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,759 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,762 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:12,762 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,762 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,763 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:12,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,763 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,763 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testInvalidSplitKey because node does not exist (not an error) 2023-02-13 00:56:12,863 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:12,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:12,877 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:12,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:12,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:12,878 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:12,878 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,878 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:12,878 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:12,879 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,879 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,881 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,881 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,881 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,881 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,882 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:12,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,883 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:12,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,883 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:12,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:12,883 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,883 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:12,884 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:12,885 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,885 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,885 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,885 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,886 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,885 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,894 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/fe598ae5d0114a7e8a605db3fb81c928 2023-02-13 00:56:12,905 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/b886a14f618b474dbb87594a36016d39 2023-02-13 00:56:12,908 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/fe598ae5d0114a7e8a605db3fb81c928 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/fe598ae5d0114a7e8a605db3fb81c928 2023-02-13 00:56:12,911 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/fe598ae5d0114a7e8a605db3fb81c928, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:12,912 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/b886a14f618b474dbb87594a36016d39 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/b886a14f618b474dbb87594a36016d39 2023-02-13 00:56:12,915 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/b886a14f618b474dbb87594a36016d39, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:12,916 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 33ms, sequenceid=58, compaction requested=true 2023-02-13 00:56:12,916 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:12,916 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:12,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:12,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:12,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:12,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:12,918 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,918 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,918 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,918 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:12,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:12,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e3f8aec[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:12,920 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:12,920 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,921 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:12,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:12,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:12,921 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:12,926 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,926 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,926 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:12,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:12,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:12,932 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:12,932 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:12,932 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:12,932 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:12,932 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:12,934 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,934 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,935 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,935 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,936 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:12,937 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:12,937 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,939 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,939 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:12,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,939 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:12,939 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:12,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:12,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:12,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:12,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:12,940 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:12,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:12,941 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:12,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,040 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:13,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:13,052 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:13,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:13,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 00:56:13,053 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 00:56:13,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 00:56:13,053 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 00:56:13,054 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,054 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,056 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,057 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,057 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,056 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,057 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,057 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,057 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 00:56:13,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:13,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:13,058 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:13,058 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:13,058 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 00:56:13,058 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:13,058 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,058 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:13,058 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:13,059 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:13,059 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:13,059 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. started... 2023-02-13 00:56:13,059 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:13,060 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,060 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,060 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,061 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:13,060 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,061 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:13,067 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/5aebfcd4f23942fd83d5cea7dd63f118 2023-02-13 00:56:13,078 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/a393ec9243c4446894272dd2c495f722 2023-02-13 00:56:13,081 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/5aebfcd4f23942fd83d5cea7dd63f118 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/5aebfcd4f23942fd83d5cea7dd63f118 2023-02-13 00:56:13,085 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/5aebfcd4f23942fd83d5cea7dd63f118, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:13,085 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/a393ec9243c4446894272dd2c495f722 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/a393ec9243c4446894272dd2c495f722 2023-02-13 00:56:13,089 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/a393ec9243c4446894272dd2c495f722, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:13,090 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 31ms, sequenceid=71, compaction requested=true 2023-02-13 00:56:13,090 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:13,090 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,090 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:13,090 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:13,090 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,090 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 00:56:13,090 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 00:56:13,092 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,092 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,092 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,092 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 00:56:13,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d54c253[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:13,094 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 00:56:13,094 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:13,096 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,096 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:13,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:13,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 00:56:13,096 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,098 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,099 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,099 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 00:56:13,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:13,102 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 00:56:13,102 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 00:56:13,102 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:13,102 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:13,102 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 00:56:13,102 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,104 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:13,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,105 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:13,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,105 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:13,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,107 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,107 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:13,108 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 00:56:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:13,110 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:13,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 00:56:13,111 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:13,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,110 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 00:56:13,111 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 00:56:13,211 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:13,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:13,218 INFO [Listener at localhost/43763] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,219 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.StoreUtils(129): cannot split hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57 because midkey is the same as first or last row 2023-02-13 00:56:13,219 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.StoreUtils(129): cannot split hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2 because midkey is the same as first or last row 2023-02-13 00:56:13,220 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegion(2744): Flushing 02c4b8e12916f904a963dc699cdc889d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:56:13,229 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/100fba967a2040be869671cad70f5012 2023-02-13 00:56:13,239 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/7ec04b34d629444f81543f218b580e10 2023-02-13 00:56:13,243 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf1/100fba967a2040be869671cad70f5012 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/100fba967a2040be869671cad70f5012 2023-02-13 00:56:13,247 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/100fba967a2040be869671cad70f5012, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:13,247 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/.tmp/cf2/7ec04b34d629444f81543f218b580e10 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/7ec04b34d629444f81543f218b580e10 2023-02-13 00:56:13,250 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/7ec04b34d629444f81543f218b580e10, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:13,251 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 02c4b8e12916f904a963dc699cdc889d in 31ms, sequenceid=82, compaction requested=true 2023-02-13 00:56:13,252 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.HRegion(2445): Flush status journal for 02c4b8e12916f904a963dc699cdc889d: 2023-02-13 00:56:13,252 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.StoreUtils(129): cannot split hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57 because midkey is the same as first or last row 2023-02-13 00:56:13,252 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41539] regionserver.StoreUtils(129): cannot split hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2 because midkey is the same as first or last row 2023-02-13 00:56:13,252 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,253 DEBUG [Listener at localhost/43763] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 00:56:13,253 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:13,254 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 00:56:13,254 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-13 00:56:13,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:56:13,257 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249773257"}]},"ts":"1676249773257"} 2023-02-13 00:56:13,258 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 00:56:13,261 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 00:56:13,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, UNASSIGN}] 2023-02-13 00:56:13,262 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, UNASSIGN 2023-02-13 00:56:13,263 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=02c4b8e12916f904a963dc699cdc889d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:13,263 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676249773263"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249773263"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249773263"}]},"ts":"1676249773263"} 2023-02-13 00:56:13,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 02c4b8e12916f904a963dc699cdc889d, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:13,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:56:13,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:13,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 02c4b8e12916f904a963dc699cdc889d, disabling compactions & flushes 2023-02-13 00:56:13,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. after waiting 0 ms 2023-02-13 00:56:13,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,420 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:56:13,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:13,421 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d. 2023-02-13 00:56:13,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 02c4b8e12916f904a963dc699cdc889d: Waiting for close lock at 1676249773416Running coprocessor pre-close hooks at 1676249773416Disabling compacts and flushes for region at 1676249773416Disabling writes for close at 1676249773416Writing region close event to WAL at 1676249773417 (+1 ms)Running coprocessor post-close hooks at 1676249773421 (+4 ms)Closed at 1676249773421 2023-02-13 00:56:13,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:13,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=02c4b8e12916f904a963dc699cdc889d, regionState=CLOSED 2023-02-13 00:56:13,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676249773423"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249773423"}]},"ts":"1676249773423"} 2023-02-13 00:56:13,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 00:56:13,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 02c4b8e12916f904a963dc699cdc889d, server=jenkins-hbase4.apache.org,41539,1676249678442 in 160 msec 2023-02-13 00:56:13,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 00:56:13,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=02c4b8e12916f904a963dc699cdc889d, UNASSIGN in 165 msec 2023-02-13 00:56:13,428 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249773428"}]},"ts":"1676249773428"} 2023-02-13 00:56:13,428 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 00:56:13,430 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 00:56:13,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 176 msec 2023-02-13 00:56:13,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 00:56:13,559 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 00:56:13,560 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-13 00:56:13,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,562 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,563 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 00:56:13,588 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:13,591 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/recovered.edits] 2023-02-13 00:56:13,594 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/0a8bea82e9004522ac4d9c92eb333e57 2023-02-13 00:56:13,595 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/100fba967a2040be869671cad70f5012 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/100fba967a2040be869671cad70f5012 2023-02-13 00:56:13,596 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/1f4f51cd63de405c89d982e64b8a559f to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/1f4f51cd63de405c89d982e64b8a559f 2023-02-13 00:56:13,597 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/5aebfcd4f23942fd83d5cea7dd63f118 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/5aebfcd4f23942fd83d5cea7dd63f118 2023-02-13 00:56:13,598 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/79f70e17e04e4ae2ab8d78df5456b63b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/79f70e17e04e4ae2ab8d78df5456b63b 2023-02-13 00:56:13,599 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/e1a9261579b84a77a5fab3715549f186 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/e1a9261579b84a77a5fab3715549f186 2023-02-13 00:56:13,600 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/fe598ae5d0114a7e8a605db3fb81c928 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf1/fe598ae5d0114a7e8a605db3fb81c928 2023-02-13 00:56:13,602 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/2cf057010cf04a129249de544eeb62c8 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/2cf057010cf04a129249de544eeb62c8 2023-02-13 00:56:13,603 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/31ee78c549c34a49b209b84d45ee6f7e to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/31ee78c549c34a49b209b84d45ee6f7e 2023-02-13 00:56:13,603 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/7ec04b34d629444f81543f218b580e10 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/7ec04b34d629444f81543f218b580e10 2023-02-13 00:56:13,604 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/9cbfa255be2346bf956c51a27ea2b4c2 2023-02-13 00:56:13,605 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/a393ec9243c4446894272dd2c495f722 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/a393ec9243c4446894272dd2c495f722 2023-02-13 00:56:13,606 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/b886a14f618b474dbb87594a36016d39 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/b886a14f618b474dbb87594a36016d39 2023-02-13 00:56:13,607 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/dfb521ff9980475fabc259843802006d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/cf2/dfb521ff9980475fabc259843802006d 2023-02-13 00:56:13,608 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d/recovered.edits/85.seqid 2023-02-13 00:56:13,609 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testInvalidSplitKey/02c4b8e12916f904a963dc699cdc889d 2023-02-13 00:56:13,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 00:56:13,611 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,612 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 00:56:13,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 00:56:13,616 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,616 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 00:56:13,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249773616"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:13,618 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 00:56:13,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 02c4b8e12916f904a963dc699cdc889d, NAME => 'testInvalidSplitKey,,1676249771643.02c4b8e12916f904a963dc699cdc889d.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 00:56:13,618 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 00:56:13,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249773618"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:13,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 00:56:13,622 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:13,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 62 msec 2023-02-13 00:56:13,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 00:56:13,664 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 00:56:13,676 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=364 (was 345) - Thread LEAK? -, OpenFileDescriptor=739 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 112), ProcessCount=167 (was 167), AvailableMemoryMB=3244 (was 3301) 2023-02-13 00:56:13,689 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=364, OpenFileDescriptor=739, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=167, AvailableMemoryMB=3244 2023-02-13 00:56:13,689 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:13,691 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:56:13,692 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:56:13,693 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:13,694 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:13,695 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:13,696 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:13,696 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 empty. 2023-02-13 00:56:13,697 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:13,697 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 00:56:13,710 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:13,711 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => e05446666901d182507c3f5c74a07087, NAME => 'testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing e05446666901d182507c3f5c74a07087, disabling compactions & flushes 2023-02-13 00:56:13,723 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. after waiting 0 ms 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:13,723 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:13,723 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for e05446666901d182507c3f5c74a07087: Waiting for close lock at 1676249773723Disabling compacts and flushes for region at 1676249773723Disabling writes for close at 1676249773723Writing region close event to WAL at 1676249773723Closed at 1676249773723 2023-02-13 00:56:13,725 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:13,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249773726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249773726"}]},"ts":"1676249773726"} 2023-02-13 00:56:13,727 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:56:13,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:13,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249773727"}]},"ts":"1676249773727"} 2023-02-13 00:56:13,728 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 00:56:13,731 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:56:13,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:56:13,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:56:13,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:56:13,732 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:56:13,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, ASSIGN}] 2023-02-13 00:56:13,733 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, ASSIGN 2023-02-13 00:56:13,733 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:56:13,884 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:56:13,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=e05446666901d182507c3f5c74a07087, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:13,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249773885"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249773885"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249773885"}]},"ts":"1676249773885"} 2023-02-13 00:56:13,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:14,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e05446666901d182507c3f5c74a07087, NAME => 'testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:56:14,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:14,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:14,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,042 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,043 DEBUG [StoreOpener-e05446666901d182507c3f5c74a07087-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1 2023-02-13 00:56:14,043 DEBUG [StoreOpener-e05446666901d182507c3f5c74a07087-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1 2023-02-13 00:56:14,044 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e05446666901d182507c3f5c74a07087 columnFamilyName cf1 2023-02-13 00:56:14,044 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] regionserver.HStore(310): Store=e05446666901d182507c3f5c74a07087/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:14,044 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,045 DEBUG [StoreOpener-e05446666901d182507c3f5c74a07087-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2 2023-02-13 00:56:14,046 DEBUG [StoreOpener-e05446666901d182507c3f5c74a07087-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2 2023-02-13 00:56:14,046 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e05446666901d182507c3f5c74a07087 columnFamilyName cf2 2023-02-13 00:56:14,046 INFO [StoreOpener-e05446666901d182507c3f5c74a07087-1] regionserver.HStore(310): Store=e05446666901d182507c3f5c74a07087/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:14,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:14,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e05446666901d182507c3f5c74a07087 2023-02-13 00:56:14,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:14,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e05446666901d182507c3f5c74a07087; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75319748, jitterRate=0.12235170602798462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:14,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e05446666901d182507c3f5c74a07087: Running coprocessor pre-open hook at 1676249774041Writing region info on filesystem at 1676249774041Initializing all the Stores at 1676249774042 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249774042Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249774042Cleaning up temporary data from old regions at 1676249774048 (+6 ms)Cleaning up detritus from prior splits at 1676249774048Running coprocessor post-open hooks at 1676249774051 (+3 ms)Region opened successfully at 1676249774051 2023-02-13 00:56:14,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087., pid=111, masterSystemTime=1676249774038 2023-02-13 00:56:14,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,054 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=e05446666901d182507c3f5c74a07087, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,054 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249774054"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249774054"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249774054"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249774054"}]},"ts":"1676249774054"} 2023-02-13 00:56:14,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 00:56:14,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 in 169 msec 2023-02-13 00:56:14,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 00:56:14,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, ASSIGN in 324 msec 2023-02-13 00:56:14,058 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:14,059 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249774059"}]},"ts":"1676249774059"} 2023-02-13 00:56:14,060 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 00:56:14,061 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:14,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 369 msec 2023-02-13 00:56:14,200 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:14,201 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,201 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,201 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:14,201 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:14,202 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,202 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,245 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,245 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,245 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,245 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,246 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,247 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,247 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,248 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:14,249 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,253 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,259 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/aa246fe98f3b48a097f5da0a8e6a469a 2023-02-13 00:56:14,270 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a 2023-02-13 00:56:14,273 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/aa246fe98f3b48a097f5da0a8e6a469a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a 2023-02-13 00:56:14,277 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:14,277 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a 2023-02-13 00:56:14,281 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:14,281 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 32ms, sequenceid=6, compaction requested=false 2023-02-13 00:56:14,281 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,282 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:14,282 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,282 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,285 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,285 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,285 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,285 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:14,287 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f579d2e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,287 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:14,287 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,288 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:14,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,289 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,289 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:14,289 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:14,290 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,290 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,290 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,291 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,291 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,291 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,291 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,292 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,292 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,292 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:14,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,296 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:14,296 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,296 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,296 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,296 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,299 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,300 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,300 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,301 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,302 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,302 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,302 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,307 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,307 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:14,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,308 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,408 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:14,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:14,419 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:14,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:14,420 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:14,420 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:14,421 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,421 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,422 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,422 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,422 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,422 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,424 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,424 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,424 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,425 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,425 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,428 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,428 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,428 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,428 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:14,428 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:14,439 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/0d6f1e9777e24572b59f737ff0ea3b10 2023-02-13 00:56:14,448 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/cd4ab905821145eba18ef487cf2c58a5 2023-02-13 00:56:14,452 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/0d6f1e9777e24572b59f737ff0ea3b10 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 2023-02-13 00:56:14,454 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:14,455 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/cd4ab905821145eba18ef487cf2c58a5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 2023-02-13 00:56:14,458 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:14,459 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 31ms, sequenceid=19, compaction requested=false 2023-02-13 00:56:14,459 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:14,459 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,459 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,459 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,459 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,459 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,459 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,470 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:14,472 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fe44c11[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,472 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:14,472 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:14,473 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,473 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,473 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:14,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,474 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,474 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,474 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:14,475 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,475 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,477 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,477 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,477 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:14,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,479 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:14,479 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,479 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,479 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,479 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,482 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,481 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,482 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,482 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,484 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,485 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,488 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,487 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,488 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:14,489 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,589 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:14,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:14,600 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:14,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:14,602 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,602 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,602 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:14,602 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:14,602 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,602 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,604 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,604 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,604 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,604 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,605 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,605 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,606 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,606 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,606 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,607 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,607 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,607 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:14,608 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,608 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,608 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,608 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,609 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,615 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/7cbf6be72ea64e648053ac35079e7b33 2023-02-13 00:56:14,624 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/26d3165a16be439bb9a7d61f28c6bf6e 2023-02-13 00:56:14,627 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/7cbf6be72ea64e648053ac35079e7b33 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 2023-02-13 00:56:14,630 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:14,631 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/26d3165a16be439bb9a7d61f28c6bf6e as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e 2023-02-13 00:56:14,633 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:14,634 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 27ms, sequenceid=32, compaction requested=true 2023-02-13 00:56:14,634 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:14,634 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,634 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,634 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,634 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,634 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,634 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,636 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,636 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,636 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,636 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:14,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3deec5eb[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,637 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:14,638 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,639 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:14,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,640 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:14,640 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,641 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,641 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:14,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,645 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:14,645 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,645 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,645 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,645 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,648 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,647 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,648 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,650 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,650 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,650 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,655 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,656 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,656 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,656 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,656 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,657 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:14,657 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,757 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:14,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:14,767 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:14,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:14,769 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,769 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,769 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:14,769 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:14,769 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,769 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,770 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,770 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,770 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,770 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,771 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,771 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,772 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,772 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,772 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,772 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,773 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,773 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,773 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,773 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:14,773 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,780 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,779 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,780 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,781 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 2023-02-13 00:56:14,794 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/e6129801ca004187a1cee04debee5ab2 2023-02-13 00:56:14,797 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 2023-02-13 00:56:14,801 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:14,802 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/e6129801ca004187a1cee04debee5ab2 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 2023-02-13 00:56:14,804 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:14,805 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 32ms, sequenceid=45, compaction requested=true 2023-02-13 00:56:14,805 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:14,805 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,807 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,807 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,807 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,807 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:14,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2633de5f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,809 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:14,809 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,810 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,811 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:14,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:14,812 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,812 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,814 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,814 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,814 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:14,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,817 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:14,817 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,817 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,817 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,817 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,819 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,820 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,820 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,822 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,822 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,822 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,824 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,824 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,824 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,824 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,825 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,824 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:14,825 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,826 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,926 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:14,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:14,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:14,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:14,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:14,940 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,940 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,940 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:14,940 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:14,940 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,940 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,942 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,942 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,942 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,942 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:14,942 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,943 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:14,944 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,944 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,944 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:14,944 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:14,948 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,949 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,952 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/a70a400f282f470db2fb9e332b74034d 2023-02-13 00:56:14,962 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/67f838b9b8934928b935a63dd473b835 2023-02-13 00:56:14,965 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/a70a400f282f470db2fb9e332b74034d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d 2023-02-13 00:56:14,968 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:14,969 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/67f838b9b8934928b935a63dd473b835 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 2023-02-13 00:56:14,972 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:14,972 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 28ms, sequenceid=58, compaction requested=true 2023-02-13 00:56:14,972 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:14,972 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:14,972 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:14,973 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:14,973 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,973 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:14,973 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:14,978 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,978 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,978 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,978 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:14,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45ee995a[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:14,980 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:14,980 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,985 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,985 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:14,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:14,985 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:14,986 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,986 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,986 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,986 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,987 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:14,989 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:14,989 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:14,989 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:14,989 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:14,989 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:14,989 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:14,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,991 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:14,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,992 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,992 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:14,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:14,994 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:14,994 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,994 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:14,997 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,997 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:14,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:14,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:14,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:14,998 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:14,999 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:14,999 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:14,999 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,075 ERROR [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid 5c1a259a-c33c-46e5-8622-2360d3fa2b07) service to localhost/127.0.0.1:41999 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:56:15,098 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:15,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:15,110 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:15,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:15,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 00:56:15,111 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:15,111 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,111 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 00:56:15,111 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 00:56:15,111 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,111 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,113 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,113 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,113 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,113 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:15,114 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:15,114 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:15,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 00:56:15,114 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,114 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:15,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:15,115 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:15,115 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. started... 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:15,115 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:15,115 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:15,116 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,116 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,116 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:15,117 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,117 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,117 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:15,123 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/383fdb420c594c0ea8b794a9077909f2 2023-02-13 00:56:15,132 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/e1c76cc06fc84425a11d56ed6b2d8927 2023-02-13 00:56:15,135 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/383fdb420c594c0ea8b794a9077909f2 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 2023-02-13 00:56:15,139 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:15,139 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/e1c76cc06fc84425a11d56ed6b2d8927 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 2023-02-13 00:56:15,142 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:15,143 INFO [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 28ms, sequenceid=71, compaction requested=true 2023-02-13 00:56:15,143 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e05446666901d182507c3f5c74a07087: 2023-02-13 00:56:15,143 DEBUG [rs(jenkins-hbase4.apache.org,34557,1676249678414)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:15,143 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:15,143 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:15,143 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,143 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 00:56:15,143 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 00:56:15,145 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,145 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,146 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,146 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 00:56:15,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3817c132[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:15,147 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 00:56:15,147 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,149 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:15,149 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:15,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:15,149 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:15,150 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:15,150 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,150 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,151 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:15,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,151 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 00:56:15,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,34557,1676249678414': 2023-02-13 00:56:15,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 00:56:15,154 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 00:56:15,154 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:15,154 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:15,154 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 00:56:15,154 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,157 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,157 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,158 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,158 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:15,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,159 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:15,159 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 00:56:15,160 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,165 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 00:56:15,165 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:15,165 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:15,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:15,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:15,166 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 00:56:15,266 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:15,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:15,524 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 00:56:15,525 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:15,525 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,526 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,527 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:15,527 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,527 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:15,527 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:15,776 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:15,776 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:15,777 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:15,778 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 00:56:15,779 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:15,779 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:15,779 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:15,779 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:15,780 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:15,780 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:15,780 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:15,780 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:15,781 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:15,781 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:15,781 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:15,781 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:15,782 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:15,782 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:15,782 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:15,782 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:15,783 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:15,783 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:15,783 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:15,783 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:15,783 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:15,784 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:15,784 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:15,784 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:15,784 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:15,785 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:15,785 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:15,785 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:15,786 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:15,786 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:15,786 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:15,786 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:15,786 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:15,787 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:15,787 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,787 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:15,787 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,787 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,788 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,788 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 00:56:15,788 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:15,789 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:15,789 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249805789 2023-02-13 00:56:15,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,791 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:15,793 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:15,794 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:15,794 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:15,794 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249835794 2023-02-13 00:56:15,794 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249895794 2023-02-13 00:56:15,794 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:15,794 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:15,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:15,796 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:15,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:15,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:16,044 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,044 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:16,044 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:16,045 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:16,045 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:16,045 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:16,045 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:16,045 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:16,045 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:16,046 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:16,047 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 00:56:16,047 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:16,047 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:16,047 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:16,047 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,048 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,049 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,050 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,051 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,052 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,052 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:16,052 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,052 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:16,052 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,052 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,052 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,052 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 00:56:16,053 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:16,053 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:16,054 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249806054 2023-02-13 00:56:16,055 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:16,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,057 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:16,057 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:16,058 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:16,058 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249836058 2023-02-13 00:56:16,058 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249896058 2023-02-13 00:56:16,058 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 00:56:16,058 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:16,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN}] 2023-02-13 00:56:16,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:16,060 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:16,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:16,308 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:16,308 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:16,309 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:16,309 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:16,310 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 00:56:16,311 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:16,311 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:16,311 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:16,311 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:16,311 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:16,312 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,313 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,314 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,315 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,315 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:16,315 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,315 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,316 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,316 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,316 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,316 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:16,317 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,317 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:16,317 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,317 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,317 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,317 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:16,318 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:16,318 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:16,319 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249806319 2023-02-13 00:56:16,320 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:16,320 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,322 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:16,322 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:16,322 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:16,323 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249836323 2023-02-13 00:56:16,323 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249896323 2023-02-13 00:56:16,323 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:16,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN}] 2023-02-13 00:56:16,323 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:16,325 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:16,326 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e05446666901d182507c3f5c74a07087, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:16,326 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249776326"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249776326"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249776326"}]},"ts":"1676249776326"} 2023-02-13 00:56:16,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:16,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e05446666901d182507c3f5c74a07087, disabling compactions & flushes 2023-02-13 00:56:16,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:16,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:16,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. after waiting 0 ms 2023-02-13 00:56:16,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:16,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing e05446666901d182507c3f5c74a07087 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:56:16,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e 2023-02-13 00:56:16,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/d6c642e5f8734f3bb703afb597e48389 2023-02-13 00:56:16,505 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e 2023-02-13 00:56:16,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:16,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/.tmp/cf2/d6c642e5f8734f3bb703afb597e48389 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 2023-02-13 00:56:16,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:16,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 32ms, sequenceid=82, compaction requested=true 2023-02-13 00:56:16,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:56:16,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:16,517 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. 2023-02-13 00:56:16,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e05446666901d182507c3f5c74a07087: Waiting for close lock at 1676249776480Running coprocessor pre-close hooks at 1676249776480Disabling compacts and flushes for region at 1676249776480Disabling writes for close at 1676249776480Obtaining lock to block concurrent updates at 1676249776480Preparing flush snapshotting stores in e05446666901d182507c3f5c74a07087 at 1676249776480Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249776481 (+1 ms)Flushing stores of testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. at 1676249776481Flushing e05446666901d182507c3f5c74a07087/cf1: creating writer at 1676249776482 (+1 ms)Flushing e05446666901d182507c3f5c74a07087/cf1: appending metadata at 1676249776484 (+2 ms)Flushing e05446666901d182507c3f5c74a07087/cf1: closing flushed file at 1676249776484Flushing e05446666901d182507c3f5c74a07087/cf2: creating writer at 1676249776495 (+11 ms)Flushing e05446666901d182507c3f5c74a07087/cf2: appending metadata at 1676249776496 (+1 ms)Flushing e05446666901d182507c3f5c74a07087/cf2: closing flushed file at 1676249776496Flushing e05446666901d182507c3f5c74a07087/cf1: reopening flushed file at 1676249776505 (+9 ms)Flushing e05446666901d182507c3f5c74a07087/cf2: reopening flushed file at 1676249776509 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e05446666901d182507c3f5c74a07087 in 32ms, sequenceid=82, compaction requested=true at 1676249776512 (+3 ms)Writing region close event to WAL at 1676249776513 (+1 ms)Running coprocessor post-close hooks at 1676249776516 (+3 ms)Closed at 1676249776517 (+1 ms) 2023-02-13 00:56:16,519 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,519 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=e05446666901d182507c3f5c74a07087, regionState=CLOSED 2023-02-13 00:56:16,519 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249776519"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249776519"}]},"ts":"1676249776519"} 2023-02-13 00:56:16,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 00:56:16,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 in 192 msec 2023-02-13 00:56:16,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 00:56:16,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN in 198 msec 2023-02-13 00:56:16,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:16,524 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:16,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:16,573 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:16,573 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:16,574 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:16,574 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:16,575 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 00:56:16,575 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:16,575 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,576 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,577 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,578 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,579 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,580 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:16,581 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,581 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,581 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:16,581 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:16,581 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:16,582 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:16,582 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:16,583 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249806583 2023-02-13 00:56:16,583 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:16,585 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,587 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:16,587 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:16,587 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:16,587 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249836587 2023-02-13 00:56:16,587 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249896587 2023-02-13 00:56:16,587 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:56:16,590 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:16,598 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=e05446666901d182507c3f5c74a07087, threads=14 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,599 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,600 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,600 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,600 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,603 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,603 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,604 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,608 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,608 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,708 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,732 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,732 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,740 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,740 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,742 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,742 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,742 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,743 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,748 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,748 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,761 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,762 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,763 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,763 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region e05446666901d182507c3f5c74a07087 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:56:16,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:16,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:16,791 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:16,838 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:16,838 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:16,839 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:16,840 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 00:56:16,840 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:16,840 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,841 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:16,842 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,842 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:16,842 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,842 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:16,842 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,843 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,844 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,845 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:16,846 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:16,846 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,847 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,847 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,847 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:16,847 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:16,847 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:16,847 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:16,848 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:16,848 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249806848 2023-02-13 00:56:16,850 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:16,850 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:16,852 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:16,853 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:16,853 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:16,853 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249836853 2023-02-13 00:56:16,853 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249896853 2023-02-13 00:56:16,853 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:16,861 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=e05446666901d182507c3f5c74a07087, threads=14 2023-02-13 00:56:16,861 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,863 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,864 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,864 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,864 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,868 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,869 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,869 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,869 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,869 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,871 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,871 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,871 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,871 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:16,911 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:56:17,040 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,048 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,055 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,055 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,055 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,061 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,068 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,068 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,068 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,070 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,096 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,096 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,098 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,099 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a for region: e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,099 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region e05446666901d182507c3f5c74a07087 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:56:17,138 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:17,143 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:17,145 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:17,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:17,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:17,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:17,353 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,353 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:17,353 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:17,354 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:17,354 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:17,354 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:17,354 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:17,354 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:17,354 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:17,355 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:17,356 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 00:56:17,356 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:17,356 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:17,357 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:17,357 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:17,357 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:17,357 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:17,358 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,358 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:17,358 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,358 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,359 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,359 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:17,359 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,359 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,360 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,360 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:17,360 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:17,360 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,360 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,361 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,361 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:17,361 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,361 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,362 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,362 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:17,362 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,362 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,362 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,363 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,363 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:17,363 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,363 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,363 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,364 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,364 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,364 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:17,364 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,365 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:17,365 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:17,365 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:17,365 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,365 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,365 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,365 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,365 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,365 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 00:56:17,366 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:17,366 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:17,367 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249807367 2023-02-13 00:56:17,368 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,368 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:17,371 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:17,371 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:17,371 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:17,371 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249837371 2023-02-13 00:56:17,371 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249897371 2023-02-13 00:56:17,371 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:17,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:17,376 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:17,376 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:17,376 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:17,622 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:17,622 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:17,623 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:17,624 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 00:56:17,624 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:17,624 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:17,624 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,625 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,626 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,627 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,628 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,628 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:17,628 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,628 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,629 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,629 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,629 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,629 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:17,630 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,630 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,630 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,630 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,631 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:17,631 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:17,632 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:17,632 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249807632 2023-02-13 00:56:17,632 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:17,634 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,636 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:17,636 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:17,636 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:17,636 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249837636 2023-02-13 00:56:17,636 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249897636 2023-02-13 00:56:17,636 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:56:17,636 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:17,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249777638"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676249777638"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676249777638"}]},"ts":"1676249777638"} 2023-02-13 00:56:17,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249777638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249777638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249777638"}]},"ts":"1676249777638"} 2023-02-13 00:56:17,639 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249777638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249777638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249777638"}]},"ts":"1676249777638"} 2023-02-13 00:56:17,640 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:17,640 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:17,640 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:17,886 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:17,887 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:17,887 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:17,888 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:17,889 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 00:56:17,889 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:17,889 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:17,889 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:17,889 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:17,890 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:17,890 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:17,890 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,890 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:17,890 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:17,891 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,892 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:17,892 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:17,892 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,892 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:17,892 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:17,893 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,894 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,894 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:17,894 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,894 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:17,895 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,895 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,895 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:17,895 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:17,896 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,896 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:17,896 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:17,896 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:17,896 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,896 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,896 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,896 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,897 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:17,897 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 00:56:17,897 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:17,898 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:17,898 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249807898 2023-02-13 00:56:17,900 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:17,900 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,902 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:17,902 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:17,902 INFO [Listener at localhost/43763] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414, table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087 2023-02-13 00:56:17,903 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=fe1acb8e5116c67ac4315939efbea6d6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:17,903 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:17,903 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:17,903 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249837903 2023-02-13 00:56:17,903 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249897903 2023-02-13 00:56:17,903 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:17,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676249777904"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676249777904"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676249777904"}]},"ts":"1676249777904"} 2023-02-13 00:56:17,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249777904"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249777904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249777904"}]},"ts":"1676249777904"} 2023-02-13 00:56:17,904 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249777904"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249777904"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249777904"}]},"ts":"1676249777904"} 2023-02-13 00:56:17,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:17,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:17,907 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:17,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:17,907 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:18,072 ERROR [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid f25dc2b5-f59c-472e-873a-01e1c6fd94f9) service to localhost/127.0.0.1:41999 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:18,154 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:18,154 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:18,155 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:18,156 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 00:56:18,156 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:18,156 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:18,157 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:18,157 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:18,157 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:18,157 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:18,158 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,158 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:18,158 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,158 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,158 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,159 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,160 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,161 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:18,162 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,162 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,162 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:18,162 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:18,162 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:18,163 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:18,163 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:18,164 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249808164 2023-02-13 00:56:18,165 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:18,165 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,167 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:18,168 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:18,168 INFO [Listener at localhost/43763] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414, table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087 2023-02-13 00:56:18,168 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=fe1acb8e5116c67ac4315939efbea6d6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:18,168 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:18,168 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:18,168 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249838168 2023-02-13 00:56:18,168 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249898168 2023-02-13 00:56:18,168 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:56:18,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:18,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN}] 2023-02-13 00:56:18,170 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:18,170 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:18,170 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:18,418 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:18,419 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:18,419 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:18,420 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:18,421 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 00:56:18,421 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:18,421 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:18,421 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:18,422 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:18,422 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:18,422 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:18,422 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,422 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,423 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,424 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,425 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:18,426 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:18,427 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,427 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,427 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:18,427 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:18,428 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 00:56:18,428 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:18,429 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:18,429 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249808429 2023-02-13 00:56:18,431 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,431 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:18,434 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:18,435 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:18,435 INFO [Listener at localhost/43763] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414, table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087 2023-02-13 00:56:18,435 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=fe1acb8e5116c67ac4315939efbea6d6, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:18,435 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 00:56:18,435 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:18,435 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249838435 2023-02-13 00:56:18,435 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249898435 2023-02-13 00:56:18,435 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 00:56:18,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN}] 2023-02-13 00:56:18,436 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:18,438 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN 2023-02-13 00:56:18,439 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:56:18,439 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN 2023-02-13 00:56:18,440 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414; forceNewPlan=false, retain=false 2023-02-13 00:56:18,589 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:56:18,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=fe1acb8e5116c67ac4315939efbea6d6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249778591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249778591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249778591"}]},"ts":"1676249778591"} 2023-02-13 00:56:18,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:18,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249778594"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249778594"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249778594"}]},"ts":"1676249778594"} 2023-02-13 00:56:18,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:18,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fe1acb8e5116c67ac4315939efbea6d6, NAME => 'testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 00:56:18,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:18,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:18,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,749 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,750 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1 2023-02-13 00:56:18,750 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1 2023-02-13 00:56:18,750 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe1acb8e5116c67ac4315939efbea6d6 columnFamilyName cf1 2023-02-13 00:56:18,764 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-bottom 2023-02-13 00:56:18,771 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-bottom 2023-02-13 00:56:18,776 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-bottom 2023-02-13 00:56:18,780 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-bottom 2023-02-13 00:56:18,786 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-bottom 2023-02-13 00:56:18,790 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-bottom 2023-02-13 00:56:18,794 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-bottom 2023-02-13 00:56:18,794 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(310): Store=fe1acb8e5116c67ac4315939efbea6d6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:18,794 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,795 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2 2023-02-13 00:56:18,795 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2 2023-02-13 00:56:18,795 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe1acb8e5116c67ac4315939efbea6d6 columnFamilyName cf2 2023-02-13 00:56:18,808 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-bottom 2023-02-13 00:56:18,812 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-bottom 2023-02-13 00:56:18,817 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-bottom 2023-02-13 00:56:18,835 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-bottom 2023-02-13 00:56:18,840 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-bottom 2023-02-13 00:56:18,843 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-bottom 2023-02-13 00:56:18,848 DEBUG [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-bottom 2023-02-13 00:56:18,848 INFO [StoreOpener-fe1acb8e5116c67ac4315939efbea6d6-1] regionserver.HStore(310): Store=fe1acb8e5116c67ac4315939efbea6d6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:18,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,850 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:18,854 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:18,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fe1acb8e5116c67ac4315939efbea6d6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65746389, jitterRate=-0.020302459597587585}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:18,855 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fe1acb8e5116c67ac4315939efbea6d6: Running coprocessor pre-open hook at 1676249778747Writing region info on filesystem at 1676249778748 (+1 ms)Initializing all the Stores at 1676249778748Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249778748Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249778748Cleaning up temporary data from old regions at 1676249778851 (+103 ms)Cleaning up detritus from prior splits at 1676249778852 (+1 ms)Running coprocessor post-open hooks at 1676249778855 (+3 ms)Region opened successfully at 1676249778855 2023-02-13 00:56:18,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6., pid=117, masterSystemTime=1676249778744 2023-02-13 00:56:18,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:18,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:18,864 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:18,864 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,864 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HStore(1912): fe1acb8e5116c67ac4315939efbea6d6/cf2 is initiating minor compaction (all files) 2023-02-13 00:56:18,864 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HRegion(2258): Starting compaction of fe1acb8e5116c67ac4315939efbea6d6/cf2 in testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,865 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/.tmp, totalSize=34.9 K 2023-02-13 00:56:18,863 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:18,865 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,865 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1912): fe1acb8e5116c67ac4315939efbea6d6/cf1 is initiating minor compaction (all files) 2023-02-13 00:56:18,865 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of fe1acb8e5116c67ac4315939efbea6d6/cf1 in testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,865 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting 5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249774195 2023-02-13 00:56:18,866 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/.tmp, totalSize=34.9 K 2023-02-13 00:56:18,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,866 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249774409 2023-02-13 00:56:18,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:18,866 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249774195 2023-02-13 00:56:18,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0fc6a32a8cef4d9b0ad89824f58b6f7a, NAME => 'testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 00:56:18,867 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting 26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249774590 2023-02-13 00:56:18,867 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249774409 2023-02-13 00:56:18,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:18,867 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249774758 2023-02-13 00:56:18,867 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249774590 2023-02-13 00:56:18,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=fe1acb8e5116c67ac4315939efbea6d6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249778867"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249778867"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249778867"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249778867"}]},"ts":"1676249778867"} 2023-02-13 00:56:18,867 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting 67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249774927 2023-02-13 00:56:18,868 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249774758 2023-02-13 00:56:18,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:18,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,868 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249775100 2023-02-13 00:56:18,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,868 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249774927 2023-02-13 00:56:18,869 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] compactions.Compactor(207): Compacting d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249775267 2023-02-13 00:56:18,869 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249775100 2023-02-13 00:56:18,870 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249775267 2023-02-13 00:56:18,871 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-13 00:56:18,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414 in 277 msec 2023-02-13 00:56:18,873 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1 2023-02-13 00:56:18,873 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1 2023-02-13 00:56:18,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN in 436 msec 2023-02-13 00:56:18,873 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0fc6a32a8cef4d9b0ad89824f58b6f7a columnFamilyName cf1 2023-02-13 00:56:18,883 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] throttle.PressureAwareThroughputController(145): fe1acb8e5116c67ac4315939efbea6d6#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:18,886 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] throttle.PressureAwareThroughputController(145): fe1acb8e5116c67ac4315939efbea6d6#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:18,898 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-top 2023-02-13 00:56:18,903 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-top 2023-02-13 00:56:18,911 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-top 2023-02-13 00:56:18,915 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-top 2023-02-13 00:56:18,915 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/.tmp/cf1/01b1e175fdd84b09891d519b078118f7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/01b1e175fdd84b09891d519b078118f7 2023-02-13 00:56:18,918 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-top 2023-02-13 00:56:18,920 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fe1acb8e5116c67ac4315939efbea6d6/cf1 of fe1acb8e5116c67ac4315939efbea6d6 into 01b1e175fdd84b09891d519b078118f7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:18,920 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for fe1acb8e5116c67ac4315939efbea6d6: 2023-02-13 00:56:18,920 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6., storeName=fe1acb8e5116c67ac4315939efbea6d6/cf1, priority=9, startTime=1676249778856; duration=0sec 2023-02-13 00:56:18,920 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:18,924 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-top 2023-02-13 00:56:18,927 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-top 2023-02-13 00:56:18,927 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(310): Store=0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:18,927 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,928 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2 2023-02-13 00:56:18,928 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2 2023-02-13 00:56:18,929 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0fc6a32a8cef4d9b0ad89824f58b6f7a columnFamilyName cf2 2023-02-13 00:56:18,942 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-top 2023-02-13 00:56:18,945 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-top 2023-02-13 00:56:18,948 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-top 2023-02-13 00:56:18,951 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-top 2023-02-13 00:56:18,953 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-top 2023-02-13 00:56:18,956 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-top 2023-02-13 00:56:18,962 DEBUG [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-top 2023-02-13 00:56:18,962 INFO [StoreOpener-0fc6a32a8cef4d9b0ad89824f58b6f7a-1] regionserver.HStore(310): Store=0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:18,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:18,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0fc6a32a8cef4d9b0ad89824f58b6f7a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66304166, jitterRate=-0.011990934610366821}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:18,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0fc6a32a8cef4d9b0ad89824f58b6f7a: Running coprocessor pre-open hook at 1676249778868Writing region info on filesystem at 1676249778868Initializing all the Stores at 1676249778869 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249778870 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249778870Cleaning up temporary data from old regions at 1676249778964 (+94 ms)Cleaning up detritus from prior splits at 1676249778965 (+1 ms)Running coprocessor post-open hooks at 1676249778967 (+2 ms)Region opened successfully at 1676249778967 2023-02-13 00:56:18,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a., pid=118, masterSystemTime=1676249778744 2023-02-13 00:56:18,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:18,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:18,968 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:18,969 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,969 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1912): 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1 is initiating minor compaction (all files) 2023-02-13 00:56:18,969 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1 in testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,969 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/.tmp, totalSize=34.9 K 2023-02-13 00:56:18,970 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249774195 2023-02-13 00:56:18,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,970 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249774409 2023-02-13 00:56:18,970 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:18,970 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249778970"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249778970"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249778970"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249778970"}]},"ts":"1676249778970"} 2023-02-13 00:56:18,970 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249774590 2023-02-13 00:56:18,971 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249774758 2023-02-13 00:56:18,971 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249774927 2023-02-13 00:56:18,972 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249775100 2023-02-13 00:56:18,972 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249775267 2023-02-13 00:56:18,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 00:56:18,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414 in 376 msec 2023-02-13 00:56:18,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-13 00:56:18,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN in 537 msec 2023-02-13 00:56:18,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 00:56:18,975 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:18,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:18,975 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 00:56:18,980 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0fc6a32a8cef4d9b0ad89824f58b6f7a#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:18,990 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/.tmp/cf1/a78d079985d1458a8939b0f181f04817 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a78d079985d1458a8939b0f181f04817 2023-02-13 00:56:18,995 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1 of 0fc6a32a8cef4d9b0ad89824f58b6f7a into a78d079985d1458a8939b0f181f04817(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:18,995 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0fc6a32a8cef4d9b0ad89824f58b6f7a: 2023-02-13 00:56:18,995 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a., storeName=0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1, priority=9, startTime=1676249778967; duration=0sec 2023-02-13 00:56:18,995 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:18,996 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:18,996 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,996 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1912): 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2 is initiating minor compaction (all files) 2023-02-13 00:56:18,996 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2 in testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:18,996 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/.tmp, totalSize=34.9 K 2023-02-13 00:56:18,997 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249774195 2023-02-13 00:56:18,997 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249774409 2023-02-13 00:56:18,997 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249774590 2023-02-13 00:56:18,998 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249774758 2023-02-13 00:56:18,998 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting 67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249774927 2023-02-13 00:56:18,998 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249775100 2023-02-13 00:56:18,999 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] compactions.Compactor(207): Compacting d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249775267 2023-02-13 00:56:19,008 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0fc6a32a8cef4d9b0ad89824f58b6f7a#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:19,026 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/.tmp/cf2/a21940fe6172458b9c012a9f3374d752 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/a21940fe6172458b9c012a9f3374d752 2023-02-13 00:56:19,030 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2 of 0fc6a32a8cef4d9b0ad89824f58b6f7a into a21940fe6172458b9c012a9f3374d752(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:19,030 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0fc6a32a8cef4d9b0ad89824f58b6f7a: 2023-02-13 00:56:19,030 INFO [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a., storeName=0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2, priority=9, startTime=1676249778967; duration=0sec 2023-02-13 00:56:19,030 DEBUG [RS:1;jenkins-hbase4:34557-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:19,186 ERROR [Listener at localhost/43763] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 00:56:19,186 INFO [Listener at localhost/43763] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 00:56:19,187 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 00:56:19,187 INFO [Listener at localhost/43763] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 00:56:19,187 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 00:56:19,188 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 00:56:19,188 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 00:56:19,188 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 00:56:19,188 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 00:56:19,189 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f8ce95eb513c26321825effddeecdacf, daughterA=fb3cb2309d059a71828c00de44bd7e7c, daughterB=cd0b9cc8d658dd293604b3f54617114c 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7978078ab37756a938cfcf56fec698c9, daughterA=3dd96bf9a4733d352bef76a275164362, daughterB=99cb1f102b60382a91299eb121580efb 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:19,190 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=ea3eaa9ea79a3bf34eba60089ea7ec8c, daughterA=411efaba7fb06518f9e9fb451db87e25, daughterB=68cbdddc658221d9acd276beb5e2889e 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=ea3eaa9ea79a3bf34eba60089ea7ec8c, ASSIGN 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:19,191 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=3bb3523ffc664edd904a25ce35edbcb3, daughterA=5081bfdc6ad0313a915da5f3b4be3a2e, daughterB=35ea1bdc294f2a77cb80ecfd357e548e 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ae67fb354b1c30a4bea3ba0a8e540f12, daughterA=2031b155b1006fc9dadf482f4464cd65, daughterB=e5a290b2c8b29817f0f0a685a173099d 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:19,192 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41539,1676249678442, table=testRollbackForSplitTableRegionWithReplica, region=dd5c9c772a71d6dc99be51911a2bcfb3 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=27681efa2c518abe27ce49fb434b6528, daughterA=fdc56a2ebdd3b8f99fb6fccab923cc9f, daughterB=876aa016cecc0895b91f4148175fb553 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:19,193 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:19,194 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 00:56:19,194 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a held the lock before restarting, call acquireLock to restore it. 2023-02-13 00:56:19,195 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,195 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,195 INFO [Listener at localhost/43763] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure e05446666901d182507c3f5c74a07087, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,195 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,196 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,196 DEBUG [Listener at localhost/43763] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 00:56:19,196 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 00:56:19,196 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 00:56:19,197 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372, table=hbase:meta, region=1588230740 2023-02-13 00:56:19,197 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676249809197 2023-02-13 00:56:19,198 DEBUG [Listener at localhost/43763] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 00:56:19,199 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,201 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=2a8283ab9ffc3d4f3875786a1eb0696d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41539,1676249678442, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442, openSeqNum=2 2023-02-13 00:56:19,201 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=e05446666901d182507c3f5c74a07087, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=2 2023-02-13 00:56:19,201 INFO [Listener at localhost/43763] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,34557,1676249678414, table=testRecoveryAndDoubleExecution, region=e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,202 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=fe1acb8e5116c67ac4315939efbea6d6, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=86 2023-02-13 00:56:19,202 DEBUG [Listener at localhost/43763] assignment.RegionStateStore(147): Load hbase:meta entry region=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,34557,1676249678414, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414, openSeqNum=86 2023-02-13 00:56:19,202 INFO [Listener at localhost/43763] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 00:56:19,202 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676249839202 2023-02-13 00:56:19,202 INFO [Listener at localhost/43763] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676249899202 2023-02-13 00:56:19,202 INFO [Listener at localhost/43763] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 00:56:19,202 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 00:56:19,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=e05446666901d182507c3f5c74a07087, daughterA=fe1acb8e5116c67ac4315939efbea6d6, daughterB=0fc6a32a8cef4d9b0ad89824f58b6f7a in 3.6770 sec 2023-02-13 00:56:19,313 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/.tmp/cf2/e6b0ba0f69a04acaa51201297af771af as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6b0ba0f69a04acaa51201297af771af 2023-02-13 00:56:19,316 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in fe1acb8e5116c67ac4315939efbea6d6/cf2 of fe1acb8e5116c67ac4315939efbea6d6 into e6b0ba0f69a04acaa51201297af771af(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:19,317 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for fe1acb8e5116c67ac4315939efbea6d6: 2023-02-13 00:56:19,317 INFO [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6., storeName=fe1acb8e5116c67ac4315939efbea6d6/cf2, priority=9, startTime=1676249778861; duration=0sec 2023-02-13 00:56:19,317 DEBUG [RS:1;jenkins-hbase4:34557-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:19,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34557] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:48492 deadline: 1676249839453, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087. is not online on jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,564 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:56:19,570 INFO [Listener at localhost/43763] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 00:56:19,574 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:56:19,582 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:19,583 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 00:56:19,583 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-13 00:56:19,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 00:56:19,587 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249779587"}]},"ts":"1676249779587"} 2023-02-13 00:56:19,588 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 00:56:19,589 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 00:56:19,590 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, UNASSIGN}] 2023-02-13 00:56:19,591 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, UNASSIGN 2023-02-13 00:56:19,591 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,591 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249779591"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249779591"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249779591"}]},"ts":"1676249779591"} 2023-02-13 00:56:19,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:19,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, UNASSIGN 2023-02-13 00:56:19,596 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=fe1acb8e5116c67ac4315939efbea6d6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:19,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249779596"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249779596"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249779596"}]},"ts":"1676249779596"} 2023-02-13 00:56:19,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414}] 2023-02-13 00:56:19,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 00:56:19,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:19,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fe1acb8e5116c67ac4315939efbea6d6, disabling compactions & flushes 2023-02-13 00:56:19,745 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:19,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:19,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. after waiting 0 ms 2023-02-13 00:56:19,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:19,745 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-bottom] to archive 2023-02-13 00:56:19,746 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:19,748 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,749 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,750 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,752 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,753 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,754 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,755 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,757 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-bottom] to archive 2023-02-13 00:56:19,757 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:19,759 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,761 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,762 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,763 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,764 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,765 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,765 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:56:19,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:19,769 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6. 2023-02-13 00:56:19,769 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fe1acb8e5116c67ac4315939efbea6d6: Waiting for close lock at 1676249779744Running coprocessor pre-close hooks at 1676249779744Disabling compacts and flushes for region at 1676249779744Disabling writes for close at 1676249779745 (+1 ms)Writing region close event to WAL at 1676249779766 (+21 ms)Running coprocessor post-close hooks at 1676249779769 (+3 ms)Closed at 1676249779769 2023-02-13 00:56:19,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:19,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0fc6a32a8cef4d9b0ad89824f58b6f7a, disabling compactions & flushes 2023-02-13 00:56:19,772 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:19,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:19,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. after waiting 0 ms 2023-02-13 00:56:19,772 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:19,772 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=fe1acb8e5116c67ac4315939efbea6d6, regionState=CLOSED 2023-02-13 00:56:19,772 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249779772"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249779772"}]},"ts":"1676249779772"} 2023-02-13 00:56:19,773 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e-top] to archive 2023-02-13 00:56:19,774 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:19,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 00:56:19,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure fe1acb8e5116c67ac4315939efbea6d6, server=jenkins-hbase4.apache.org,34557,1676249678414 in 177 msec 2023-02-13 00:56:19,776 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=fe1acb8e5116c67ac4315939efbea6d6, UNASSIGN in 186 msec 2023-02-13 00:56:19,778 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,779 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,781 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,782 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,783 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,784 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,793 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389-top] to archive 2023-02-13 00:56:19,793 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:19,795 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,796 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,797 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,798 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,799 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,799 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,800 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,803 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:56:19,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:19,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a. 2023-02-13 00:56:19,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0fc6a32a8cef4d9b0ad89824f58b6f7a: Waiting for close lock at 1676249779772Running coprocessor pre-close hooks at 1676249779772Disabling compacts and flushes for region at 1676249779772Disabling writes for close at 1676249779772Writing region close event to WAL at 1676249779801 (+29 ms)Running coprocessor post-close hooks at 1676249779804 (+3 ms)Closed at 1676249779804 2023-02-13 00:56:19,806 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=0fc6a32a8cef4d9b0ad89824f58b6f7a, regionState=CLOSED 2023-02-13 00:56:19,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676249779806"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249779806"}]},"ts":"1676249779806"} 2023-02-13 00:56:19,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 00:56:19,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 0fc6a32a8cef4d9b0ad89824f58b6f7a, server=jenkins-hbase4.apache.org,34557,1676249678414 in 215 msec 2023-02-13 00:56:19,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-02-13 00:56:19,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=0fc6a32a8cef4d9b0ad89824f58b6f7a, UNASSIGN in 219 msec 2023-02-13 00:56:19,810 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249779810"}]},"ts":"1676249779810"} 2023-02-13 00:56:19,811 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 00:56:19,813 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 00:56:19,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 230 msec 2023-02-13 00:56:19,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 00:56:19,888 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 00:56:19,889 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-13 00:56:19,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 00:56:19,896 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,896 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:19,896 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,898 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/recovered.edits] 2023-02-13 00:56:19,898 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/recovered.edits] 2023-02-13 00:56:19,898 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/recovered.edits] 2023-02-13 00:56:19,901 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/01b1e175fdd84b09891d519b078118f7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf1/01b1e175fdd84b09891d519b078118f7 2023-02-13 00:56:19,901 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/.regioninfo to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/.regioninfo 2023-02-13 00:56:19,902 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/0d6f1e9777e24572b59f737ff0ea3b10 2023-02-13 00:56:19,903 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/383fdb420c594c0ea8b794a9077909f2 2023-02-13 00:56:19,904 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6b0ba0f69a04acaa51201297af771af to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6b0ba0f69a04acaa51201297af771af 2023-02-13 00:56:19,904 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,905 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/50cd2da6ecee4c79b8943b7113f6bcf3 2023-02-13 00:56:19,906 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/7cbf6be72ea64e648053ac35079e7b33 2023-02-13 00:56:19,906 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,907 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/.regioninfo to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/.regioninfo 2023-02-13 00:56:19,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/a70a400f282f470db2fb9e332b74034d 2023-02-13 00:56:19,907 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,908 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/aa246fe98f3b48a097f5da0a8e6a469a 2023-02-13 00:56:19,909 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,910 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e 2023-02-13 00:56:19,910 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,910 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/0d6f1e9777e24572b59f737ff0ea3b10.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,912 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/383fdb420c594c0ea8b794a9077909f2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,913 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,913 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/26d3165a16be439bb9a7d61f28c6bf6e 2023-02-13 00:56:19,913 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/50cd2da6ecee4c79b8943b7113f6bcf3.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,914 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a 2023-02-13 00:56:19,915 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/7cbf6be72ea64e648053ac35079e7b33.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,916 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/67f838b9b8934928b935a63dd473b835 2023-02-13 00:56:19,916 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,916 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/a70a400f282f470db2fb9e332b74034d.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,917 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/cd4ab905821145eba18ef487cf2c58a5 2023-02-13 00:56:19,918 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,918 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/aa246fe98f3b48a097f5da0a8e6a469a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,919 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/d6c642e5f8734f3bb703afb597e48389 2023-02-13 00:56:19,919 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf1/d8d2945d9a494d1cbff7f75e0ef21e0e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,919 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e1c76cc06fc84425a11d56ed6b2d8927 2023-02-13 00:56:19,921 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,922 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/cf2/e6129801ca004187a1cee04debee5ab2 2023-02-13 00:56:19,923 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/26d3165a16be439bb9a7d61f28c6bf6e.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,923 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,925 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/5fd1b62c5bcd4e41aa9ac6ddf58c6a8a.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,925 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,925 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087/recovered.edits/85.seqid 2023-02-13 00:56:19,928 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,928 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/67f838b9b8934928b935a63dd473b835.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,928 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,930 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/cd4ab905821145eba18ef487cf2c58a5.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,931 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a78d079985d1458a8939b0f181f04817 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf1/a78d079985d1458a8939b0f181f04817 2023-02-13 00:56:19,931 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/d6c642e5f8734f3bb703afb597e48389.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,932 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/e1c76cc06fc84425a11d56ed6b2d8927.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,933 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/a21940fe6172458b9c012a9f3374d752 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/cf2/a21940fe6172458b9c012a9f3374d752 2023-02-13 00:56:19,933 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/fe1acb8e5116c67ac4315939efbea6d6/cf2/e6129801ca004187a1cee04debee5ab2.e05446666901d182507c3f5c74a07087 2023-02-13 00:56:19,935 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a/recovered.edits/90.seqid 2023-02-13 00:56:19,935 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6/recovered.edits/90.seqid 2023-02-13 00:56:19,936 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/0fc6a32a8cef4d9b0ad89824f58b6f7a 2023-02-13 00:56:19,936 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testRecoveryAndDoubleExecution/fe1acb8e5116c67ac4315939efbea6d6 2023-02-13 00:56:19,936 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 00:56:19,938 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,940 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 00:56:19,941 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 00:56:19,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,942 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 00:56:19,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249779942"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:19,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249779942"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:19,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249779942"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:19,944 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:56:19,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e05446666901d182507c3f5c74a07087, NAME => 'testRecoveryAndDoubleExecution,,1676249773692.e05446666901d182507c3f5c74a07087.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => fe1acb8e5116c67ac4315939efbea6d6, NAME => 'testRecoveryAndDoubleExecution,,1676249775525.fe1acb8e5116c67ac4315939efbea6d6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 0fc6a32a8cef4d9b0ad89824f58b6f7a, NAME => 'testRecoveryAndDoubleExecution,41,1676249775525.0fc6a32a8cef4d9b0ad89824f58b6f7a.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 00:56:19,944 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 00:56:19,944 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249779944"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:19,945 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 00:56:19,947 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 00:56:19,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 58 msec 2023-02-13 00:56:19,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 00:56:19,993 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 00:56:20,007 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=372 (was 364) - Thread LEAK? -, OpenFileDescriptor=850 (was 739) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=104 (was 112), ProcessCount=167 (was 167), AvailableMemoryMB=3227 (was 3244) 2023-02-13 00:56:20,020 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=372, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=104, ProcessCount=167, AvailableMemoryMB=3226 2023-02-13 00:56:20,021 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:20,023 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:56:20,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:56:20,024 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,025 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:20,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:20,027 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,028 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 empty. 2023-02-13 00:56:20,028 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,028 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 00:56:20,036 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:20,037 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 82f270d3080bc732470ea7684fa6df58, NAME => 'testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 82f270d3080bc732470ea7684fa6df58, disabling compactions & flushes 2023-02-13 00:56:20,044 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. after waiting 0 ms 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,044 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,044 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 82f270d3080bc732470ea7684fa6df58: Waiting for close lock at 1676249780044Disabling compacts and flushes for region at 1676249780044Disabling writes for close at 1676249780044Writing region close event to WAL at 1676249780044Closed at 1676249780044 2023-02-13 00:56:20,046 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:20,046 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249780046"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249780046"}]},"ts":"1676249780046"} 2023-02-13 00:56:20,047 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:56:20,048 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:20,048 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249780048"}]},"ts":"1676249780048"} 2023-02-13 00:56:20,049 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 00:56:20,052 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:56:20,053 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:56:20,053 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:56:20,053 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:56:20,053 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:56:20,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, ASSIGN}] 2023-02-13 00:56:20,054 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, ASSIGN 2023-02-13 00:56:20,055 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:56:20,205 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:56:20,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=82f270d3080bc732470ea7684fa6df58, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249780206"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249780206"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249780206"}]},"ts":"1676249780206"} 2023-02-13 00:56:20,207 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 82f270d3080bc732470ea7684fa6df58, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:20,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 82f270d3080bc732470ea7684fa6df58, NAME => 'testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:56:20,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:20,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:20,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,362 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,363 DEBUG [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1 2023-02-13 00:56:20,364 DEBUG [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1 2023-02-13 00:56:20,364 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82f270d3080bc732470ea7684fa6df58 columnFamilyName cf1 2023-02-13 00:56:20,364 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] regionserver.HStore(310): Store=82f270d3080bc732470ea7684fa6df58/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:20,364 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,365 DEBUG [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2 2023-02-13 00:56:20,365 DEBUG [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2 2023-02-13 00:56:20,366 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 82f270d3080bc732470ea7684fa6df58 columnFamilyName cf2 2023-02-13 00:56:20,366 INFO [StoreOpener-82f270d3080bc732470ea7684fa6df58-1] regionserver.HStore(310): Store=82f270d3080bc732470ea7684fa6df58/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:20,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:20,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:20,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:20,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 82f270d3080bc732470ea7684fa6df58; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59882172, jitterRate=-0.1076861023902893}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:20,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 82f270d3080bc732470ea7684fa6df58: Running coprocessor pre-open hook at 1676249780362Writing region info on filesystem at 1676249780362Initializing all the Stores at 1676249780362Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249780362Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249780362Cleaning up temporary data from old regions at 1676249780367 (+5 ms)Cleaning up detritus from prior splits at 1676249780368 (+1 ms)Running coprocessor post-open hooks at 1676249780371 (+3 ms)Region opened successfully at 1676249780371 2023-02-13 00:56:20,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58., pid=127, masterSystemTime=1676249780359 2023-02-13 00:56:20,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,373 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=82f270d3080bc732470ea7684fa6df58, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,373 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249780373"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249780373"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249780373"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249780373"}]},"ts":"1676249780373"} 2023-02-13 00:56:20,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 00:56:20,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 82f270d3080bc732470ea7684fa6df58, server=jenkins-hbase4.apache.org,41539,1676249678442 in 167 msec 2023-02-13 00:56:20,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 00:56:20,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, ASSIGN in 323 msec 2023-02-13 00:56:20,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:20,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249780378"}]},"ts":"1676249780378"} 2023-02-13 00:56:20,379 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 00:56:20,381 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:20,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 357 msec 2023-02-13 00:56:20,532 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:20,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,533 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:20,533 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,534 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:20,534 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:20,534 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,534 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,536 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:20,536 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:20,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:20,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:20,536 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:20,536 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:20,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:20,537 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,537 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,538 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:20,542 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,542 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:20,542 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,542 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:20,542 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:20,542 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:20,543 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,544 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:20,544 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:20,544 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:20,544 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:20,545 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:20,545 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:56:20,545 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:20,547 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,547 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,547 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:20,562 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/f0ee575e689645c997b468f7fecc92ab 2023-02-13 00:56:20,977 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/4d9c798534d74df9bae3b0dafc0c4196 2023-02-13 00:56:20,981 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/f0ee575e689645c997b468f7fecc92ab as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab 2023-02-13 00:56:20,984 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:20,985 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/4d9c798534d74df9bae3b0dafc0c4196 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196 2023-02-13 00:56:20,988 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:20,989 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 444ms, sequenceid=6, compaction requested=false 2023-02-13 00:56:20,989 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:20,989 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:20,989 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:20,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:20,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:20,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:20,990 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:20,991 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,991 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:20,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:20,991 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,991 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:20,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:20,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:20,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:20,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@739d8a46[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:20,993 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:20,993 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:20,996 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,996 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:20,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:20,996 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:20,997 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,997 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:20,997 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:20,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:20,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:20,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:20,997 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:20,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:20,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:20,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:20,999 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:20,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:20,999 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:20,999 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:20,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,003 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,003 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,003 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,003 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,003 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,005 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,007 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,007 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,008 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,008 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,009 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,014 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,014 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,014 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,015 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,115 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,125 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,127 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,127 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,127 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:21,127 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:21,127 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,127 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,129 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,129 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,129 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,131 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,131 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:21,131 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:21,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,135 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,139 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/a8a5756fcf0943a7a99d98dfce324408 2023-02-13 00:56:21,148 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/bad666c773f04f5095dd0e83e4293d41 2023-02-13 00:56:21,151 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/a8a5756fcf0943a7a99d98dfce324408 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408 2023-02-13 00:56:21,154 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:21,154 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/bad666c773f04f5095dd0e83e4293d41 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41 2023-02-13 00:56:21,157 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:21,158 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 27ms, sequenceid=19, compaction requested=false 2023-02-13 00:56:21,158 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:21,158 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,158 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,159 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,159 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,160 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,160 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:21,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b64734c[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:21,161 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,162 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,162 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,162 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,163 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:21,163 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,163 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:21,165 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,165 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,165 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,165 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,166 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,166 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,166 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,166 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,167 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,167 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,169 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,169 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,169 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,171 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,171 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,172 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,173 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,173 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,174 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,178 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,178 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,177 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,177 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,178 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,179 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,179 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,179 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,179 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,179 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,278 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,290 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,291 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,291 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:21,291 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:21,291 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,291 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,292 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,292 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,292 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,293 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,294 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:21,295 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,295 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,295 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:21,295 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:21,295 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,296 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,296 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,296 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,296 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,296 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,306 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/dbcbfbd2ba284e0d8822e964be015764 2023-02-13 00:56:21,316 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 2023-02-13 00:56:21,319 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/dbcbfbd2ba284e0d8822e964be015764 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764 2023-02-13 00:56:21,323 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:21,323 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 2023-02-13 00:56:21,326 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:21,327 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 32ms, sequenceid=32, compaction requested=true 2023-02-13 00:56:21,327 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:21,327 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:21,327 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,328 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,328 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,330 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,330 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:21,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7af10733[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:21,332 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,333 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:21,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,333 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,338 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,338 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,339 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,339 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,339 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,342 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,342 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,342 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,342 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,342 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,344 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,345 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,346 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,346 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,347 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,349 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,349 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,349 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,349 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,350 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,351 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,351 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,450 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,461 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,463 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:21,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:21,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,463 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,464 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,464 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,465 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,465 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,466 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,466 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,466 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:21,467 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,467 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,467 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,467 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,467 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:21,467 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,467 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:21,469 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,470 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,470 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,470 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,479 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/387ef87f64d8493f905dbc6b4250f591 2023-02-13 00:56:21,492 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/13e442d328aa4cb5a68265ed7b4b7d07 2023-02-13 00:56:21,495 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/387ef87f64d8493f905dbc6b4250f591 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591 2023-02-13 00:56:21,498 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:21,499 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/13e442d328aa4cb5a68265ed7b4b7d07 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07 2023-02-13 00:56:21,502 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:21,502 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 35ms, sequenceid=45, compaction requested=true 2023-02-13 00:56:21,503 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:21,503 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,503 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,504 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,504 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:21,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ce9470e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,506 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:21,506 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,507 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,507 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,507 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,508 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,508 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:21,508 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,511 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,512 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,512 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,514 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,515 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,515 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,516 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,517 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,517 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,518 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,519 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,519 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,520 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,525 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,525 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,525 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,525 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,526 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,626 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,637 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,638 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,638 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,638 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:21,638 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:21,639 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,639 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,640 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,640 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,640 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,641 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,641 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,642 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,642 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,642 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,642 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:21,643 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,645 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,645 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,645 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,645 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,645 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,650 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 2023-02-13 00:56:21,659 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/477b6bcc8e9e42b284cae06808130e4d 2023-02-13 00:56:21,662 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 2023-02-13 00:56:21,665 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:21,666 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/477b6bcc8e9e42b284cae06808130e4d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d 2023-02-13 00:56:21,668 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:21,669 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 27ms, sequenceid=58, compaction requested=true 2023-02-13 00:56:21,669 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:21,669 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,669 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:21,669 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:21,669 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,669 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,669 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,671 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,671 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,671 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,671 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:21,672 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63393d6f[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,672 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:21,673 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,673 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,673 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,673 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,674 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,674 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:21,675 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,675 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,675 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,675 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,676 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,675 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,676 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,676 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,676 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,678 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,678 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,678 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,678 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,678 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,681 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,681 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,682 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,683 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,684 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,684 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,686 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,686 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,686 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,687 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,787 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,798 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:21,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:21,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,799 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,799 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,799 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 00:56:21,799 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 00:56:21,800 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,800 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,801 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,801 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,801 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,802 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,803 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 00:56:21,803 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:21,803 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:21,803 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. started... 2023-02-13 00:56:21,803 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 00:56:21,804 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,804 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,804 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:21,804 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,805 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,807 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,807 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,807 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,811 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/02adde600c9d460b988f0673c1ea9c82 2023-02-13 00:56:21,823 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/87a8aa16431a444591c4aeb1a9a4169a 2023-02-13 00:56:21,826 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/02adde600c9d460b988f0673c1ea9c82 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82 2023-02-13 00:56:21,829 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:21,830 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/87a8aa16431a444591c4aeb1a9a4169a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a 2023-02-13 00:56:21,833 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:21,834 INFO [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 30ms, sequenceid=71, compaction requested=true 2023-02-13 00:56:21,834 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 82f270d3080bc732470ea7684fa6df58: 2023-02-13 00:56:21,834 DEBUG [rs(jenkins-hbase4.apache.org,41539,1676249678442)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 00:56:21,835 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 00:56:21,836 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,836 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,837 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,837 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:21,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 00:56:21,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b2fdc57[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:21,838 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 00:56:21,839 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,843 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:21,843 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,845 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,845 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 00:56:21,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,41539,1676249678442': 2023-02-13 00:56:21,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 00:56:21,849 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 00:56:21,849 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:21,849 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:21,849 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,849 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:21,853 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,853 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:21,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,853 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:21,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,855 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,855 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:21,856 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,859 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 00:56:21,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:21,859 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:21,859 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:21,860 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,860 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:21,860 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:21,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,860 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,861 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,861 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:21,861 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:21,861 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,861 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 00:56:21,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:21,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:21,961 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:21,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:21,971 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,972 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=82f270d3080bc732470ea7684fa6df58, daughterA=4ea217554ffc3828d79ccc688247ddf9, daughterB=49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:21,973 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=82f270d3080bc732470ea7684fa6df58, daughterA=4ea217554ffc3828d79ccc688247ddf9, daughterB=49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:21,973 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=82f270d3080bc732470ea7684fa6df58, daughterA=4ea217554ffc3828d79ccc688247ddf9, daughterB=49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:21,973 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=82f270d3080bc732470ea7684fa6df58, daughterA=4ea217554ffc3828d79ccc688247ddf9, daughterB=49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:21,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, UNASSIGN}] 2023-02-13 00:56:21,976 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, UNASSIGN 2023-02-13 00:56:21,977 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=82f270d3080bc732470ea7684fa6df58, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:21,977 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249781977"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249781977"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249781977"}]},"ts":"1676249781977"} 2023-02-13 00:56:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 82f270d3080bc732470ea7684fa6df58, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:22,018 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:56:22,131 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,131 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 82f270d3080bc732470ea7684fa6df58, disabling compactions & flushes 2023-02-13 00:56:22,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:22,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:22,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. after waiting 0 ms 2023-02-13 00:56:22,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:22,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 82f270d3080bc732470ea7684fa6df58 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:56:22,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/27535c4f257748c2bb920b285409dc98 2023-02-13 00:56:22,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/ee8eae9f6d4844a3ac86dda814fa176b 2023-02-13 00:56:22,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf1/27535c4f257748c2bb920b285409dc98 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98 2023-02-13 00:56:22,164 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:22,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.tmp/cf2/ee8eae9f6d4844a3ac86dda814fa176b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b 2023-02-13 00:56:22,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:22,169 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 37ms, sequenceid=82, compaction requested=true 2023-02-13 00:56:22,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:56:22,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:22,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. 2023-02-13 00:56:22,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 82f270d3080bc732470ea7684fa6df58: Waiting for close lock at 1676249782131Running coprocessor pre-close hooks at 1676249782131Disabling compacts and flushes for region at 1676249782131Disabling writes for close at 1676249782132 (+1 ms)Obtaining lock to block concurrent updates at 1676249782132Preparing flush snapshotting stores in 82f270d3080bc732470ea7684fa6df58 at 1676249782132Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249782132Flushing stores of testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. at 1676249782132Flushing 82f270d3080bc732470ea7684fa6df58/cf1: creating writer at 1676249782135 (+3 ms)Flushing 82f270d3080bc732470ea7684fa6df58/cf1: appending metadata at 1676249782136 (+1 ms)Flushing 82f270d3080bc732470ea7684fa6df58/cf1: closing flushed file at 1676249782136Flushing 82f270d3080bc732470ea7684fa6df58/cf2: creating writer at 1676249782145 (+9 ms)Flushing 82f270d3080bc732470ea7684fa6df58/cf2: appending metadata at 1676249782146 (+1 ms)Flushing 82f270d3080bc732470ea7684fa6df58/cf2: closing flushed file at 1676249782146Flushing 82f270d3080bc732470ea7684fa6df58/cf1: reopening flushed file at 1676249782161 (+15 ms)Flushing 82f270d3080bc732470ea7684fa6df58/cf2: reopening flushed file at 1676249782165 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 82f270d3080bc732470ea7684fa6df58 in 37ms, sequenceid=82, compaction requested=true at 1676249782169 (+4 ms)Writing region close event to WAL at 1676249782170 (+1 ms)Running coprocessor post-close hooks at 1676249782173 (+3 ms)Closed at 1676249782174 (+1 ms) 2023-02-13 00:56:22,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,176 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=82f270d3080bc732470ea7684fa6df58, regionState=CLOSED 2023-02-13 00:56:22,176 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249782176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249782176"}]},"ts":"1676249782176"} 2023-02-13 00:56:22,178 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 00:56:22,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 82f270d3080bc732470ea7684fa6df58, server=jenkins-hbase4.apache.org,41539,1676249678442 in 199 msec 2023-02-13 00:56:22,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 00:56:22,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=82f270d3080bc732470ea7684fa6df58, UNASSIGN in 204 msec 2023-02-13 00:56:22,188 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=82f270d3080bc732470ea7684fa6df58, threads=14 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,189 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,190 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,190 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,190 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,190 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,193 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5, top=false 2023-02-13 00:56:22,193 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196, top=false 2023-02-13 00:56:22,193 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07, top=false 2023-02-13 00:56:22,194 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591, top=false 2023-02-13 00:56:22,195 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98, top=false 2023-02-13 00:56:22,198 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5, top=false 2023-02-13 00:56:22,198 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764, top=false 2023-02-13 00:56:22,199 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d, top=false 2023-02-13 00:56:22,199 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41, top=false 2023-02-13 00:56:22,200 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82, top=false 2023-02-13 00:56:22,200 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab, top=false 2023-02-13 00:56:22,200 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408, top=false 2023-02-13 00:56:22,201 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b, top=false 2023-02-13 00:56:22,202 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a, top=false 2023-02-13 00:56:22,204 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,204 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,208 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,208 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,210 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,212 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,212 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,214 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,218 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,219 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,218 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,219 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,220 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,221 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,221 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,223 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,223 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,225 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,225 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,225 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,226 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,226 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,226 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,228 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,228 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/.splits/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82 for child: 4ea217554ffc3828d79ccc688247ddf9, parent: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,228 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,228 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,231 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82 for region: 82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:22,231 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 82f270d3080bc732470ea7684fa6df58 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 00:56:22,253 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:22,254 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:22,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676249782255"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676249782255"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676249782255"}]},"ts":"1676249782255"} 2023-02-13 00:56:22,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782255"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249782255"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249782255"}]},"ts":"1676249782255"} 2023-02-13 00:56:22,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782255"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249782255"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249782255"}]},"ts":"1676249782255"} 2023-02-13 00:56:22,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, ASSIGN}] 2023-02-13 00:56:22,259 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, ASSIGN 2023-02-13 00:56:22,259 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:56:22,260 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, ASSIGN 2023-02-13 00:56:22,260 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41539,1676249678442; forceNewPlan=false, retain=false 2023-02-13 00:56:22,410 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:56:22,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=49c76ed3db5227044866a71aac25e5a5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782411"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249782411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249782411"}]},"ts":"1676249782411"} 2023-02-13 00:56:22,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 49c76ed3db5227044866a71aac25e5a5, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:22,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=4ea217554ffc3828d79ccc688247ddf9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,417 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782416"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249782416"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249782416"}]},"ts":"1676249782416"} 2023-02-13 00:56:22,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 4ea217554ffc3828d79ccc688247ddf9, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:22,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4ea217554ffc3828d79ccc688247ddf9, NAME => 'testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 00:56:22,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:22,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:22,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,568 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,569 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1 2023-02-13 00:56:22,569 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1 2023-02-13 00:56:22,569 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ea217554ffc3828d79ccc688247ddf9 columnFamilyName cf1 2023-02-13 00:56:22,573 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82 2023-02-13 00:56:22,576 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98 2023-02-13 00:56:22,578 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591 2023-02-13 00:56:22,580 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408 2023-02-13 00:56:22,583 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764 2023-02-13 00:56:22,585 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab 2023-02-13 00:56:22,587 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5 2023-02-13 00:56:22,587 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(310): Store=4ea217554ffc3828d79ccc688247ddf9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:22,587 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,588 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2 2023-02-13 00:56:22,588 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2 2023-02-13 00:56:22,588 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4ea217554ffc3828d79ccc688247ddf9 columnFamilyName cf2 2023-02-13 00:56:22,592 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07 2023-02-13 00:56:22,594 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d 2023-02-13 00:56:22,597 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196 2023-02-13 00:56:22,599 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5 2023-02-13 00:56:22,601 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a 2023-02-13 00:56:22,603 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41 2023-02-13 00:56:22,606 DEBUG [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b 2023-02-13 00:56:22,606 INFO [StoreOpener-4ea217554ffc3828d79ccc688247ddf9-1] regionserver.HStore(310): Store=4ea217554ffc3828d79ccc688247ddf9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:22,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:22,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4ea217554ffc3828d79ccc688247ddf9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68705442, jitterRate=0.023790866136550903}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:22,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4ea217554ffc3828d79ccc688247ddf9: Running coprocessor pre-open hook at 1676249782567Writing region info on filesystem at 1676249782567Initializing all the Stores at 1676249782568 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249782568Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249782568Cleaning up temporary data from old regions at 1676249782608 (+40 ms)Cleaning up detritus from prior splits at 1676249782608Running coprocessor post-open hooks at 1676249782610 (+2 ms)Region opened successfully at 1676249782610 2023-02-13 00:56:22,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9., pid=134, masterSystemTime=1676249782564 2023-02-13 00:56:22,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:22,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:22,611 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:22,611 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:22,612 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 00:56:22,612 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 00:56:22,612 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1912): 4ea217554ffc3828d79ccc688247ddf9/cf1 is initiating minor compaction (all files) 2023-02-13 00:56:22,612 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1912): 4ea217554ffc3828d79ccc688247ddf9/cf2 is initiating minor compaction (all files) 2023-02-13 00:56:22,612 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4ea217554ffc3828d79ccc688247ddf9/cf1 in testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,612 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 4ea217554ffc3828d79ccc688247ddf9/cf2 in testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,612 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/.tmp, totalSize=34.9 K 2023-02-13 00:56:22,612 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/.tmp, totalSize=34.9 K 2023-02-13 00:56:22,612 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249780527 2023-02-13 00:56:22,613 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249781116 2023-02-13 00:56:22,613 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249780527 2023-02-13 00:56:22,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:22,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 49c76ed3db5227044866a71aac25e5a5, NAME => 'testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 00:56:22,613 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249781280 2023-02-13 00:56:22,613 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249781116 2023-02-13 00:56:22,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:22,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:22,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,614 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=4ea217554ffc3828d79ccc688247ddf9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,614 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249781280 2023-02-13 00:56:22,614 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782614"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249782614"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249782614"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249782614"}]},"ts":"1676249782614"} 2023-02-13 00:56:22,614 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249781452 2023-02-13 00:56:22,615 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249781452 2023-02-13 00:56:22,615 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,615 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249781628 2023-02-13 00:56:22,615 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249781628 2023-02-13 00:56:22,616 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249781789 2023-02-13 00:56:22,616 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249781789 2023-02-13 00:56:22,616 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249781962 2023-02-13 00:56:22,616 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249781962 2023-02-13 00:56:22,619 DEBUG [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf1 2023-02-13 00:56:22,619 DEBUG [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf1 2023-02-13 00:56:22,619 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 49c76ed3db5227044866a71aac25e5a5 columnFamilyName cf1 2023-02-13 00:56:22,620 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] regionserver.HStore(310): Store=49c76ed3db5227044866a71aac25e5a5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:22,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-13 00:56:22,620 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 4ea217554ffc3828d79ccc688247ddf9, server=jenkins-hbase4.apache.org,41539,1676249678442 in 200 msec 2023-02-13 00:56:22,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, ASSIGN in 362 msec 2023-02-13 00:56:22,622 DEBUG [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf2 2023-02-13 00:56:22,622 DEBUG [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf2 2023-02-13 00:56:22,622 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 49c76ed3db5227044866a71aac25e5a5 columnFamilyName cf2 2023-02-13 00:56:22,623 INFO [StoreOpener-49c76ed3db5227044866a71aac25e5a5-1] regionserver.HStore(310): Store=49c76ed3db5227044866a71aac25e5a5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:22,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:22,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:22,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 49c76ed3db5227044866a71aac25e5a5; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66818644, jitterRate=-0.004324615001678467}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:22,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 49c76ed3db5227044866a71aac25e5a5: Running coprocessor pre-open hook at 1676249782614Writing region info on filesystem at 1676249782614Initializing all the Stores at 1676249782615 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249782615Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249782615Cleaning up temporary data from old regions at 1676249782625 (+10 ms)Cleaning up detritus from prior splits at 1676249782626 (+1 ms)Running coprocessor post-open hooks at 1676249782629 (+3 ms)Region opened successfully at 1676249782629 2023-02-13 00:56:22,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5., pid=133, masterSystemTime=1676249782564 2023-02-13 00:56:22,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:22,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:22,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=49c76ed3db5227044866a71aac25e5a5, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782631"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249782631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249782631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249782631"}]},"ts":"1676249782631"} 2023-02-13 00:56:22,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-13 00:56:22,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 49c76ed3db5227044866a71aac25e5a5, server=jenkins-hbase4.apache.org,41539,1676249678442 in 220 msec 2023-02-13 00:56:22,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 00:56:22,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, ASSIGN in 377 msec 2023-02-13 00:56:22,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=82f270d3080bc732470ea7684fa6df58, daughterA=4ea217554ffc3828d79ccc688247ddf9, daughterB=49c76ed3db5227044866a71aac25e5a5 in 665 msec 2023-02-13 00:56:22,648 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4ea217554ffc3828d79ccc688247ddf9#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:22,649 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] throttle.PressureAwareThroughputController(145): 4ea217554ffc3828d79ccc688247ddf9#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:22,658 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/.tmp/cf1/2662a94864e34aefb44b067a16098e02 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/2662a94864e34aefb44b067a16098e02 2023-02-13 00:56:22,660 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/.tmp/cf2/fe3ad487cf8847729f2f7e2afc2308d9 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/fe3ad487cf8847729f2f7e2afc2308d9 2023-02-13 00:56:22,662 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4ea217554ffc3828d79ccc688247ddf9/cf1 of 4ea217554ffc3828d79ccc688247ddf9 into 2662a94864e34aefb44b067a16098e02(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:22,662 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4ea217554ffc3828d79ccc688247ddf9: 2023-02-13 00:56:22,662 INFO [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9., storeName=4ea217554ffc3828d79ccc688247ddf9/cf1, priority=9, startTime=1676249782611; duration=0sec 2023-02-13 00:56:22,662 DEBUG [RS:2;jenkins-hbase4:41539-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:22,664 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4ea217554ffc3828d79ccc688247ddf9/cf2 of 4ea217554ffc3828d79ccc688247ddf9 into fe3ad487cf8847729f2f7e2afc2308d9(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:22,664 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4ea217554ffc3828d79ccc688247ddf9: 2023-02-13 00:56:22,664 INFO [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9., storeName=4ea217554ffc3828d79ccc688247ddf9/cf2, priority=9, startTime=1676249782611; duration=0sec 2023-02-13 00:56:22,664 DEBUG [RS:2;jenkins-hbase4:41539-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:22,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41539] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:40408 deadline: 1676249842723, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58. is not online on jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,832 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 00:56:22,835 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:22,837 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 00:56:22,837 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-13 00:56:22,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:22,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 00:56:22,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249782840"}]},"ts":"1676249782840"} 2023-02-13 00:56:22,841 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 00:56:22,842 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 00:56:22,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, UNASSIGN}] 2023-02-13 00:56:22,843 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, UNASSIGN 2023-02-13 00:56:22,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=49c76ed3db5227044866a71aac25e5a5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,844 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782843"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249782843"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249782843"}]},"ts":"1676249782843"} 2023-02-13 00:56:22,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 49c76ed3db5227044866a71aac25e5a5, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:22,846 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, UNASSIGN 2023-02-13 00:56:22,847 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=4ea217554ffc3828d79ccc688247ddf9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:22,847 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249782847"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249782847"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249782847"}]},"ts":"1676249782847"} 2023-02-13 00:56:22,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 4ea217554ffc3828d79ccc688247ddf9, server=jenkins-hbase4.apache.org,41539,1676249678442}] 2023-02-13 00:56:22,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 00:56:22,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:22,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4ea217554ffc3828d79ccc688247ddf9, disabling compactions & flushes 2023-02-13 00:56:22,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. after waiting 0 ms 2023-02-13 00:56:22,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:22,996 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98] to archive 2023-02-13 00:56:22,997 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:22,998 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-f0ee575e689645c997b468f7fecc92ab 2023-02-13 00:56:22,999 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-a8a5756fcf0943a7a99d98dfce324408 2023-02-13 00:56:23,000 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-dbcbfbd2ba284e0d8822e964be015764 2023-02-13 00:56:23,001 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-387ef87f64d8493f905dbc6b4250f591 2023-02-13 00:56:23,002 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-fdd8f5544c1b455b9b8f4469c4a43da5 2023-02-13 00:56:23,003 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-02adde600c9d460b988f0673c1ea9c82 2023-02-13 00:56:23,004 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-27535c4f257748c2bb920b285409dc98 2023-02-13 00:56:23,005 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b] to archive 2023-02-13 00:56:23,005 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:23,007 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-4d9c798534d74df9bae3b0dafc0c4196 2023-02-13 00:56:23,007 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-bad666c773f04f5095dd0e83e4293d41 2023-02-13 00:56:23,008 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-873fe0b3070b4e65be8a505f6f9cd5e5 2023-02-13 00:56:23,009 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-13e442d328aa4cb5a68265ed7b4b7d07 2023-02-13 00:56:23,010 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-477b6bcc8e9e42b284cae06808130e4d 2023-02-13 00:56:23,011 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-87a8aa16431a444591c4aeb1a9a4169a 2023-02-13 00:56:23,012 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/testSplitTableRegionEmptyDaughter=82f270d3080bc732470ea7684fa6df58-ee8eae9f6d4844a3ac86dda814fa176b 2023-02-13 00:56:23,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:56:23,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:23,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9. 2023-02-13 00:56:23,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4ea217554ffc3828d79ccc688247ddf9: Waiting for close lock at 1676249782995Running coprocessor pre-close hooks at 1676249782995Disabling compacts and flushes for region at 1676249782995Disabling writes for close at 1676249782995Writing region close event to WAL at 1676249783013 (+18 ms)Running coprocessor post-close hooks at 1676249783018 (+5 ms)Closed at 1676249783019 (+1 ms) 2023-02-13 00:56:23,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:23,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:23,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 49c76ed3db5227044866a71aac25e5a5, disabling compactions & flushes 2023-02-13 00:56:23,021 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:23,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:23,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. after waiting 0 ms 2023-02-13 00:56:23,021 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:23,021 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=4ea217554ffc3828d79ccc688247ddf9, regionState=CLOSED 2023-02-13 00:56:23,021 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249783021"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249783021"}]},"ts":"1676249783021"} 2023-02-13 00:56:23,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 00:56:23,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 00:56:23,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 4ea217554ffc3828d79ccc688247ddf9, server=jenkins-hbase4.apache.org,41539,1676249678442 in 174 msec 2023-02-13 00:56:23,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:23,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5. 2023-02-13 00:56:23,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 49c76ed3db5227044866a71aac25e5a5: Waiting for close lock at 1676249783020Running coprocessor pre-close hooks at 1676249783020Disabling compacts and flushes for region at 1676249783020Disabling writes for close at 1676249783021 (+1 ms)Writing region close event to WAL at 1676249783021Running coprocessor post-close hooks at 1676249783024 (+3 ms)Closed at 1676249783024 2023-02-13 00:56:23,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4ea217554ffc3828d79ccc688247ddf9, UNASSIGN in 182 msec 2023-02-13 00:56:23,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:23,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=49c76ed3db5227044866a71aac25e5a5, regionState=CLOSED 2023-02-13 00:56:23,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676249783025"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249783025"}]},"ts":"1676249783025"} 2023-02-13 00:56:23,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 00:56:23,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 49c76ed3db5227044866a71aac25e5a5, server=jenkins-hbase4.apache.org,41539,1676249678442 in 182 msec 2023-02-13 00:56:23,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-13 00:56:23,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=49c76ed3db5227044866a71aac25e5a5, UNASSIGN in 185 msec 2023-02-13 00:56:23,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249783029"}]},"ts":"1676249783029"} 2023-02-13 00:56:23,029 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 00:56:23,032 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 00:56:23,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-02-13 00:56:23,073 ERROR [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid f25dc2b5-f59c-472e-873a-01e1c6fd94f9) service to localhost/127.0.0.1:41999 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:56:23,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 00:56:23,142 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 00:56:23,142 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,143 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,145 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 00:56:23,149 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:23,149 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:23,149 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:23,151 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/recovered.edits] 2023-02-13 00:56:23,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/recovered.edits] 2023-02-13 00:56:23,151 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/recovered.edits] 2023-02-13 00:56:23,153 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/02adde600c9d460b988f0673c1ea9c82 2023-02-13 00:56:23,153 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/2662a94864e34aefb44b067a16098e02 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf1/2662a94864e34aefb44b067a16098e02 2023-02-13 00:56:23,154 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/27535c4f257748c2bb920b285409dc98 2023-02-13 00:56:23,155 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/recovered.edits/88.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5/recovered.edits/88.seqid 2023-02-13 00:56:23,156 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/49c76ed3db5227044866a71aac25e5a5 2023-02-13 00:56:23,156 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/fe3ad487cf8847729f2f7e2afc2308d9 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/cf2/fe3ad487cf8847729f2f7e2afc2308d9 2023-02-13 00:56:23,156 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/387ef87f64d8493f905dbc6b4250f591 2023-02-13 00:56:23,157 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/a8a5756fcf0943a7a99d98dfce324408 2023-02-13 00:56:23,158 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9/recovered.edits/90.seqid 2023-02-13 00:56:23,158 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/dbcbfbd2ba284e0d8822e964be015764 2023-02-13 00:56:23,159 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/4ea217554ffc3828d79ccc688247ddf9 2023-02-13 00:56:23,160 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/f0ee575e689645c997b468f7fecc92ab 2023-02-13 00:56:23,160 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf1/fdd8f5544c1b455b9b8f4469c4a43da5 2023-02-13 00:56:23,163 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/13e442d328aa4cb5a68265ed7b4b7d07 2023-02-13 00:56:23,164 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/477b6bcc8e9e42b284cae06808130e4d 2023-02-13 00:56:23,165 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/4d9c798534d74df9bae3b0dafc0c4196 2023-02-13 00:56:23,166 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/873fe0b3070b4e65be8a505f6f9cd5e5 2023-02-13 00:56:23,167 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/87a8aa16431a444591c4aeb1a9a4169a 2023-02-13 00:56:23,167 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/bad666c773f04f5095dd0e83e4293d41 2023-02-13 00:56:23,168 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/cf2/ee8eae9f6d4844a3ac86dda814fa176b 2023-02-13 00:56:23,170 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58/recovered.edits/85.seqid 2023-02-13 00:56:23,171 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegionEmptyDaughter/82f270d3080bc732470ea7684fa6df58 2023-02-13 00:56:23,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 00:56:23,172 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,174 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 00:56:23,175 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 00:56:23,176 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,176 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 00:56:23,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249783176"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:23,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249783176"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:23,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249783176"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:23,177 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:56:23,178 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 82f270d3080bc732470ea7684fa6df58, NAME => 'testSplitTableRegionEmptyDaughter,,1676249780024.82f270d3080bc732470ea7684fa6df58.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 4ea217554ffc3828d79ccc688247ddf9, NAME => 'testSplitTableRegionEmptyDaughter,,1676249781971.4ea217554ffc3828d79ccc688247ddf9.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 49c76ed3db5227044866a71aac25e5a5, NAME => 'testSplitTableRegionEmptyDaughter,71,1676249781971.49c76ed3db5227044866a71aac25e5a5.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 00:56:23,178 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 00:56:23,178 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249783178"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:23,179 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 00:56:23,180 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 00:56:23,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 39 msec 2023-02-13 00:56:23,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 00:56:23,247 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 00:56:23,262 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=372 (was 372), OpenFileDescriptor=850 (was 850), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 104), ProcessCount=167 (was 167), AvailableMemoryMB=3218 (was 3226) 2023-02-13 00:56:23,275 INFO [Listener at localhost/43763] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=372, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=167, AvailableMemoryMB=3217 2023-02-13 00:56:23,275 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:23,278 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 00:56:23,278 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-13 00:56:23,279 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 00:56:23,280 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 00:56:23,281 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 00:56:23,282 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,282 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 empty. 2023-02-13 00:56:23,283 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 00:56:23,292 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 00:56:23,294 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7a35765982dbe0f471877a25d912db99, NAME => 'testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 7a35765982dbe0f471877a25d912db99, disabling compactions & flushes 2023-02-13 00:56:23,301 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. after waiting 0 ms 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,301 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,301 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 7a35765982dbe0f471877a25d912db99: Waiting for close lock at 1676249783301Disabling compacts and flushes for region at 1676249783301Disabling writes for close at 1676249783301Writing region close event to WAL at 1676249783301Closed at 1676249783301 2023-02-13 00:56:23,303 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 00:56:23,303 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249783303"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249783303"}]},"ts":"1676249783303"} 2023-02-13 00:56:23,305 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 00:56:23,305 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 00:56:23,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249783305"}]},"ts":"1676249783305"} 2023-02-13 00:56:23,306 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 00:56:23,310 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-13 00:56:23,311 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 00:56:23,311 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 00:56:23,311 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 00:56:23,311 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 00:56:23,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, ASSIGN}] 2023-02-13 00:56:23,312 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, ASSIGN 2023-02-13 00:56:23,312 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,43727,1676249678372; forceNewPlan=false, retain=false 2023-02-13 00:56:23,463 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 00:56:23,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=7a35765982dbe0f471877a25d912db99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249783464"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249783464"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249783464"}]},"ts":"1676249783464"} 2023-02-13 00:56:23,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 7a35765982dbe0f471877a25d912db99, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:23,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7a35765982dbe0f471877a25d912db99, NAME => 'testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.', STARTKEY => '', ENDKEY => ''} 2023-02-13 00:56:23,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:23,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:23,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,623 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,624 DEBUG [StoreOpener-7a35765982dbe0f471877a25d912db99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1 2023-02-13 00:56:23,624 DEBUG [StoreOpener-7a35765982dbe0f471877a25d912db99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1 2023-02-13 00:56:23,625 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a35765982dbe0f471877a25d912db99 columnFamilyName cf1 2023-02-13 00:56:23,625 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] regionserver.HStore(310): Store=7a35765982dbe0f471877a25d912db99/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:23,625 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,626 DEBUG [StoreOpener-7a35765982dbe0f471877a25d912db99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2 2023-02-13 00:56:23,626 DEBUG [StoreOpener-7a35765982dbe0f471877a25d912db99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2 2023-02-13 00:56:23,626 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a35765982dbe0f471877a25d912db99 columnFamilyName cf2 2023-02-13 00:56:23,627 INFO [StoreOpener-7a35765982dbe0f471877a25d912db99-1] regionserver.HStore(310): Store=7a35765982dbe0f471877a25d912db99/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:23,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:23,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:23,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 00:56:23,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7a35765982dbe0f471877a25d912db99; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61265390, jitterRate=-0.08707454800605774}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:23,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7a35765982dbe0f471877a25d912db99: Running coprocessor pre-open hook at 1676249783622Writing region info on filesystem at 1676249783622Initializing all the Stores at 1676249783623 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249783623Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249783623Cleaning up temporary data from old regions at 1676249783628 (+5 ms)Cleaning up detritus from prior splits at 1676249783628Running coprocessor post-open hooks at 1676249783631 (+3 ms)Region opened successfully at 1676249783631 2023-02-13 00:56:23,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99., pid=143, masterSystemTime=1676249783616 2023-02-13 00:56:23,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=7a35765982dbe0f471877a25d912db99, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249783634"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249783634"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249783634"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249783634"}]},"ts":"1676249783634"} 2023-02-13 00:56:23,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 00:56:23,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 7a35765982dbe0f471877a25d912db99, server=jenkins-hbase4.apache.org,43727,1676249678372 in 170 msec 2023-02-13 00:56:23,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 00:56:23,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, ASSIGN in 325 msec 2023-02-13 00:56:23,638 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 00:56:23,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249783638"}]},"ts":"1676249783638"} 2023-02-13 00:56:23,639 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 00:56:23,640 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 00:56:23,641 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 362 msec 2023-02-13 00:56:23,786 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:23,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:23,787 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:23,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:23,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:23,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,787 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,788 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,788 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,789 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:23,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,790 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:23,790 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:23,790 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,791 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,791 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,791 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:23,791 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 00:56:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,794 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,794 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,794 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,794 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,794 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,801 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/1b4b15ec3e44480cb16ab29c86bbb765 2023-02-13 00:56:23,817 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/d818351635d1415699fba25d955955b7 2023-02-13 00:56:23,820 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/1b4b15ec3e44480cb16ab29c86bbb765 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765 2023-02-13 00:56:23,823 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:23,823 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/d818351635d1415699fba25d955955b7 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7 2023-02-13 00:56:23,826 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:23,827 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 36ms, sequenceid=6, compaction requested=false 2023-02-13 00:56:23,827 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 00:56:23,827 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:23,827 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,827 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:23,827 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:23,827 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,827 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,827 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:23,829 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,829 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,829 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,829 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:23,831 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@79159a9e[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:23,831 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:23,831 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:23,832 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,832 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:23,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:23,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:23,832 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:23,835 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,835 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:23,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,835 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,835 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,838 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:23,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:23,840 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:23,840 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:23,840 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:23,841 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:23,841 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:23,842 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,843 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,843 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,845 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:23,845 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:23,845 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:23,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:23,848 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:23,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:23,848 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:23,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,849 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:23,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:23,949 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:23,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:23,959 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:23,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:23,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:23,961 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:23,961 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,961 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:23,961 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:23,961 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,961 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,962 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,962 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:23,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:23,963 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,963 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:23,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:23,964 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:23,965 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,965 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:23,965 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:23,965 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,965 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:23,965 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:23,965 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:23,966 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,967 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,967 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,967 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,967 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:23,967 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:23,976 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 2023-02-13 00:56:23,988 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/bec0a1192c714c01839b9306c3d0710c 2023-02-13 00:56:23,991 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 2023-02-13 00:56:23,994 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:23,995 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/bec0a1192c714c01839b9306c3d0710c as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c 2023-02-13 00:56:23,997 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 00:56:23,998 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 33ms, sequenceid=19, compaction requested=false 2023-02-13 00:56:23,998 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:23,998 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:23,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:23,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:23,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:23,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:23,998 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,001 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,001 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:24,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ff27385[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:24,003 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:24,003 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,004 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,004 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,004 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:24,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,005 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,006 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,006 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,007 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,007 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,007 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:24,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:24,010 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:24,010 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:24,010 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:24,010 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:24,010 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,012 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,013 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,013 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,015 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,015 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,015 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:24,018 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:24,018 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:24,018 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,019 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,072 ERROR [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid 4e877f44-49cc-41c6-9a12-77167167aa83) service to localhost/127.0.0.1:41999 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 00:56:24,119 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:24,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:24,129 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:24,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:24,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:24,131 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:24,131 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,131 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:24,131 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:24,131 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,131 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,132 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,133 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,134 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,133 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,134 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,134 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,134 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,134 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,134 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,134 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,134 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,134 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,134 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:24,135 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,135 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:24,136 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,140 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,143 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/72eca8eda66445e1960e6694485350c6 2023-02-13 00:56:24,553 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/2869547754ef431390468c7117c1e2c0 2023-02-13 00:56:24,557 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/72eca8eda66445e1960e6694485350c6 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6 2023-02-13 00:56:24,560 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:24,560 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/2869547754ef431390468c7117c1e2c0 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0 2023-02-13 00:56:24,563 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 00:56:24,564 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 429ms, sequenceid=32, compaction requested=true 2023-02-13 00:56:24,564 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:24,564 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:24,564 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:24,564 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:24,564 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,564 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,564 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,565 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,565 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,566 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,566 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:24,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@658aed51[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:24,567 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:24,567 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,568 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,569 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,568 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,569 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:24,570 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,570 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,570 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,572 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,572 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,572 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,572 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,572 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,572 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,573 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:24,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:24,575 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:24,575 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:24,575 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:24,575 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:24,575 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,576 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,577 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,577 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,577 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,577 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,579 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,579 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,579 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:24,582 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:24,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,582 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,582 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:24,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,583 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,683 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:24,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:24,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:24,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:24,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:24,694 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:24,694 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,694 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:24,694 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:24,695 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,695 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,697 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,697 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,697 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,698 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,698 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,699 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,699 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,699 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:24,699 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:24,700 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,701 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,701 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,701 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,701 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,701 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,710 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/3106e5b287444248bb029a6eec28ab9d 2023-02-13 00:56:24,720 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/c15b3cd105484fd2a9fa197e8be194f4 2023-02-13 00:56:24,723 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/3106e5b287444248bb029a6eec28ab9d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d 2023-02-13 00:56:24,726 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:24,727 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/c15b3cd105484fd2a9fa197e8be194f4 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4 2023-02-13 00:56:24,730 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 00:56:24,730 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 31ms, sequenceid=45, compaction requested=true 2023-02-13 00:56:24,730 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:24,730 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:24,731 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:24,731 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:24,731 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,731 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,731 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,732 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:24,734 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10b258d3[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:24,734 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:24,734 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,736 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,737 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:24,736 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:24,737 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:24,737 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,738 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,738 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,739 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,739 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,739 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:24,743 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:24,743 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:24,744 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:24,744 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:24,744 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:24,744 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:24,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:24,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,748 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:24,751 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,752 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:24,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:24,753 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,754 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,754 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,761 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,761 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:24,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:24,761 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:24,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,763 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,763 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:24,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:24,764 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:24,864 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:24,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:24,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:24,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:24,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:24,876 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:24,876 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,876 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:24,876 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:24,877 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,877 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,878 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,878 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,878 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,878 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:24,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,879 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:24,879 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:24,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,879 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:24,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:24,880 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,880 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,881 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:24,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,880 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:24,881 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:24,881 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:24,881 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:24,881 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:24,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:24,883 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,296 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/e45b337d824043d6b8564584ed3b432e 2023-02-13 00:56:25,714 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/38c5eac27f0142adb269dc2da3a35bba 2023-02-13 00:56:25,718 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/e45b337d824043d6b8564584ed3b432e as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e 2023-02-13 00:56:25,721 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:25,721 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/38c5eac27f0142adb269dc2da3a35bba as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba 2023-02-13 00:56:25,724 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 00:56:25,725 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 844ms, sequenceid=58, compaction requested=true 2023-02-13 00:56:25,725 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:25,725 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:25,725 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:25,726 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,727 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,727 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,727 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:25,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45a6f131[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:25,729 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:25,729 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,730 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,730 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,731 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,731 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,731 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,731 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:25,731 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,731 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,732 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,732 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,733 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,733 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,733 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:25,735 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:25,735 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:25,735 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:25,735 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:25,735 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:25,735 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,737 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,737 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,738 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,738 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,740 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,741 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,741 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:25,744 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:25,743 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,743 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,744 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,744 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:25,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,744 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,744 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:25,744 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,745 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,845 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:25,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:25,856 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-13 00:56:25,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 00:56:25,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 00:56:25,857 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 00:56:25,857 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,857 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 00:56:25,857 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 00:56:25,858 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,858 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,860 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,860 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,860 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,860 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,860 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 00:56:25,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:25,861 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:25,861 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41539,1676249678442' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:25,862 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:25,862 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,862 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:25,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:25,862 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34557,1676249678414' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:25,865 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,866 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,866 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,866 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 00:56:25,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 00:56:25,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 00:56:25,867 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 00:56:25,867 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:25,867 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. started... 2023-02-13 00:56:25,867 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 00:56:25,880 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/fc0a5bc1d34c435ba0f7a003b4de229d 2023-02-13 00:56:25,894 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/e2816a2f0f57411e875c92d073ebec7a 2023-02-13 00:56:25,897 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/fc0a5bc1d34c435ba0f7a003b4de229d as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d 2023-02-13 00:56:25,901 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:25,901 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/e2816a2f0f57411e875c92d073ebec7a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a 2023-02-13 00:56:25,904 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 00:56:25,905 INFO [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 38ms, sequenceid=71, compaction requested=true 2023-02-13 00:56:25,905 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7a35765982dbe0f471877a25d912db99: 2023-02-13 00:56:25,905 DEBUG [rs(jenkins-hbase4.apache.org,43727,1676249678372)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:25,905 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 00:56:25,905 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 00:56:25,905 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,905 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 00:56:25,905 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 00:56:25,907 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,907 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,907 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,907 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 00:56:25,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,43727,1676249678372' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 00:56:25,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1914dc5b[Count = 0] remaining members to acquire global barrier 2023-02-13 00:56:25,909 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 00:56:25,909 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,911 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,911 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,912 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,912 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 00:56:25,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41539,1676249678442' in zk 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,43727,1676249678372' in zk 2023-02-13 00:56:25,912 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,34557,1676249678414' in zk 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,916 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,41539,1676249678442' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,43727,1676249678372' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,916 DEBUG [member: 'jenkins-hbase4.apache.org,34557,1676249678414' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 00:56:25,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,43727,1676249678372': 2023-02-13 00:56:25,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,43727,1676249678372' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 00:56:25,919 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 00:56:25,919 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 00:56:25,919 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 00:56:25,919 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 00:56:25,919 INFO [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 00:56:25,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 00:56:25,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,922 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,923 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,923 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 00:56:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,925 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 00:56:25,925 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 00:56:25,925 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 DEBUG [(jenkins-hbase4.apache.org,43619,1676249677220)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 00:56:25,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 00:56:25,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 00:56:25,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 00:56:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,928 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 00:56:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 00:56:25,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 00:56:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 00:56:25,929 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 00:56:26,029 DEBUG [Listener at localhost/43763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 00:56:26,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 00:56:26,036 DEBUG [Listener at localhost/43763] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:26,037 DEBUG [Listener at localhost/43763] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a35765982dbe0f471877a25d912db99, daughterA=35d4233ba23f09eba4fa74bccc3ef40a, daughterB=03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:26,038 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a35765982dbe0f471877a25d912db99, daughterA=35d4233ba23f09eba4fa74bccc3ef40a, daughterB=03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:26,038 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a35765982dbe0f471877a25d912db99, daughterA=35d4233ba23f09eba4fa74bccc3ef40a, daughterB=03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:26,038 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a35765982dbe0f471877a25d912db99, daughterA=35d4233ba23f09eba4fa74bccc3ef40a, daughterB=03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:26,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, UNASSIGN}] 2023-02-13 00:56:26,040 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, UNASSIGN 2023-02-13 00:56:26,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=7a35765982dbe0f471877a25d912db99, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:26,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249786041"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249786041"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249786041"}]},"ts":"1676249786041"} 2023-02-13 00:56:26,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 7a35765982dbe0f471877a25d912db99, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:26,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7a35765982dbe0f471877a25d912db99, disabling compactions & flushes 2023-02-13 00:56:26,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:26,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:26,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. after waiting 0 ms 2023-02-13 00:56:26,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:26,194 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 7a35765982dbe0f471877a25d912db99 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 00:56:26,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/f56e82f8eb2b43928c6aa41327414720 2023-02-13 00:56:26,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/73ba3e7ae97c442b8f48ae4af86f6649 2023-02-13 00:56:26,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf1/f56e82f8eb2b43928c6aa41327414720 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720 2023-02-13 00:56:26,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:26,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/.tmp/cf2/73ba3e7ae97c442b8f48ae4af86f6649 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649 2023-02-13 00:56:26,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 00:56:26,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 36ms, sequenceid=82, compaction requested=true 2023-02-13 00:56:26,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 00:56:26,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:26,236 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. 2023-02-13 00:56:26,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7a35765982dbe0f471877a25d912db99: Waiting for close lock at 1676249786194Running coprocessor pre-close hooks at 1676249786194Disabling compacts and flushes for region at 1676249786194Disabling writes for close at 1676249786194Obtaining lock to block concurrent updates at 1676249786194Preparing flush snapshotting stores in 7a35765982dbe0f471877a25d912db99 at 1676249786194Finished memstore snapshotting testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676249786194Flushing stores of testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. at 1676249786195 (+1 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf1: creating writer at 1676249786196 (+1 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf1: appending metadata at 1676249786198 (+2 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf1: closing flushed file at 1676249786198Flushing 7a35765982dbe0f471877a25d912db99/cf2: creating writer at 1676249786209 (+11 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf2: appending metadata at 1676249786211 (+2 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf2: closing flushed file at 1676249786211Flushing 7a35765982dbe0f471877a25d912db99/cf1: reopening flushed file at 1676249786221 (+10 ms)Flushing 7a35765982dbe0f471877a25d912db99/cf2: reopening flushed file at 1676249786226 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7a35765982dbe0f471877a25d912db99 in 36ms, sequenceid=82, compaction requested=true at 1676249786230 (+4 ms)Writing region close event to WAL at 1676249786232 (+2 ms)Running coprocessor post-close hooks at 1676249786236 (+4 ms)Closed at 1676249786236 2023-02-13 00:56:26,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,238 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=7a35765982dbe0f471877a25d912db99, regionState=CLOSED 2023-02-13 00:56:26,238 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249786238"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249786238"}]},"ts":"1676249786238"} 2023-02-13 00:56:26,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 00:56:26,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 7a35765982dbe0f471877a25d912db99, server=jenkins-hbase4.apache.org,43727,1676249678372 in 197 msec 2023-02-13 00:56:26,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 00:56:26,242 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=7a35765982dbe0f471877a25d912db99, UNASSIGN in 201 msec 2023-02-13 00:56:26,253 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=7a35765982dbe0f471877a25d912db99, threads=14 2023-02-13 00:56:26,254 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,254 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,254 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,258 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,258 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,258 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,258 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,258 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,259 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,419 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,432 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,440 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,441 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,448 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,448 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,455 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,456 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,462 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,464 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,472 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,474 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,752 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,757 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 for region: 7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:26,757 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 7a35765982dbe0f471877a25d912db99 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 00:56:26,790 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:26,792 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 00:56:26,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676249786793"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676249786793"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676249786793"}]},"ts":"1676249786793"} 2023-02-13 00:56:26,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249786793"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249786793"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249786793"}]},"ts":"1676249786793"} 2023-02-13 00:56:26,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249786793"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249786793"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249786793"}]},"ts":"1676249786793"} 2023-02-13 00:56:26,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, ASSIGN}] 2023-02-13 00:56:26,796 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, ASSIGN 2023-02-13 00:56:26,799 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43727,1676249678372; forceNewPlan=false, retain=false 2023-02-13 00:56:26,799 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, ASSIGN 2023-02-13 00:56:26,800 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,43727,1676249678372; forceNewPlan=false, retain=false 2023-02-13 00:56:26,949 INFO [jenkins-hbase4:43619] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 00:56:26,951 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=35d4233ba23f09eba4fa74bccc3ef40a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:26,951 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249786951"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249786951"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249786951"}]},"ts":"1676249786951"} 2023-02-13 00:56:26,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 35d4233ba23f09eba4fa74bccc3ef40a, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:26,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=03c1550deb9567a2c004d092a0fcbbc1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:26,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249786954"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249786954"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249786954"}]},"ts":"1676249786954"} 2023-02-13 00:56:26,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 03c1550deb9567a2c004d092a0fcbbc1, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:27,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 35d4233ba23f09eba4fa74bccc3ef40a, NAME => 'testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 00:56:27,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:27,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:27,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,107 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,108 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1 2023-02-13 00:56:27,108 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1 2023-02-13 00:56:27,108 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35d4233ba23f09eba4fa74bccc3ef40a columnFamilyName cf1 2023-02-13 00:56:27,120 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-bottom 2023-02-13 00:56:27,123 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-bottom 2023-02-13 00:56:27,127 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-bottom 2023-02-13 00:56:27,130 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-bottom 2023-02-13 00:56:27,133 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-bottom 2023-02-13 00:56:27,137 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-bottom 2023-02-13 00:56:27,140 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-bottom 2023-02-13 00:56:27,141 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(310): Store=35d4233ba23f09eba4fa74bccc3ef40a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:27,141 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,141 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2 2023-02-13 00:56:27,142 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2 2023-02-13 00:56:27,142 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 35d4233ba23f09eba4fa74bccc3ef40a columnFamilyName cf2 2023-02-13 00:56:27,153 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-bottom 2023-02-13 00:56:27,156 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-bottom 2023-02-13 00:56:27,159 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-bottom 2023-02-13 00:56:27,162 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-bottom 2023-02-13 00:56:27,165 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-bottom 2023-02-13 00:56:27,168 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-bottom 2023-02-13 00:56:27,171 DEBUG [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-bottom 2023-02-13 00:56:27,171 INFO [StoreOpener-35d4233ba23f09eba4fa74bccc3ef40a-1] regionserver.HStore(310): Store=35d4233ba23f09eba4fa74bccc3ef40a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:27,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:27,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 35d4233ba23f09eba4fa74bccc3ef40a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72648799, jitterRate=0.08255146443843842}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:27,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 35d4233ba23f09eba4fa74bccc3ef40a: Running coprocessor pre-open hook at 1676249787107Writing region info on filesystem at 1676249787107Initializing all the Stores at 1676249787107Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249787107Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249787107Cleaning up temporary data from old regions at 1676249787174 (+67 ms)Cleaning up detritus from prior splits at 1676249787174Running coprocessor post-open hooks at 1676249787176 (+2 ms)Region opened successfully at 1676249787176 2023-02-13 00:56:27,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a., pid=149, masterSystemTime=1676249787103 2023-02-13 00:56:27,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:27,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:27,178 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:27,178 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:27,178 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,178 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1912): 35d4233ba23f09eba4fa74bccc3ef40a/cf1 is initiating minor compaction (all files) 2023-02-13 00:56:27,178 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,178 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 35d4233ba23f09eba4fa74bccc3ef40a/cf1 in testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,178 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HStore(1912): 35d4233ba23f09eba4fa74bccc3ef40a/cf2 is initiating minor compaction (all files) 2023-02-13 00:56:27,178 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 35d4233ba23f09eba4fa74bccc3ef40a/cf2 in testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,179 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/.tmp, totalSize=34.9 K 2023-02-13 00:56:27,179 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-bottom] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/.tmp, totalSize=34.9 K 2023-02-13 00:56:27,179 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249783782 2023-02-13 00:56:27,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,180 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676249783782 2023-02-13 00:56:27,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 03c1550deb9567a2c004d092a0fcbbc1, NAME => 'testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 00:56:27,180 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249783950 2023-02-13 00:56:27,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 00:56:27,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 00:56:27,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,181 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676249783950 2023-02-13 00:56:27,181 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249784120 2023-02-13 00:56:27,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=35d4233ba23f09eba4fa74bccc3ef40a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:27,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787181"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249787181"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249787181"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249787181"}]},"ts":"1676249787181"} 2023-02-13 00:56:27,181 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting 2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676249784120 2023-02-13 00:56:27,181 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249784684 2023-02-13 00:56:27,181 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,182 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676249784684 2023-02-13 00:56:27,182 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249784866 2023-02-13 00:56:27,183 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249785846 2023-02-13 00:56:27,183 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1 2023-02-13 00:56:27,183 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1 2023-02-13 00:56:27,183 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting 38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676249784866 2023-02-13 00:56:27,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-13 00:56:27,183 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03c1550deb9567a2c004d092a0fcbbc1 columnFamilyName cf1 2023-02-13 00:56:27,184 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 35d4233ba23f09eba4fa74bccc3ef40a, server=jenkins-hbase4.apache.org,43727,1676249678372 in 230 msec 2023-02-13 00:56:27,183 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249786030 2023-02-13 00:56:27,184 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676249785846 2023-02-13 00:56:27,185 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] compactions.Compactor(207): Compacting 73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676249786030 2023-02-13 00:56:27,186 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, ASSIGN in 388 msec 2023-02-13 00:56:27,195 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 35d4233ba23f09eba4fa74bccc3ef40a#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:27,196 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-top 2023-02-13 00:56:27,196 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] throttle.PressureAwareThroughputController(145): 35d4233ba23f09eba4fa74bccc3ef40a#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:27,199 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-top 2023-02-13 00:56:27,208 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-top 2023-02-13 00:56:27,214 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-top 2023-02-13 00:56:27,214 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/.tmp/cf1/4a79047474964908a2020aaedc0445fe as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/4a79047474964908a2020aaedc0445fe 2023-02-13 00:56:27,217 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-top 2023-02-13 00:56:27,219 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35d4233ba23f09eba4fa74bccc3ef40a/cf1 of 35d4233ba23f09eba4fa74bccc3ef40a into 4a79047474964908a2020aaedc0445fe(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:27,219 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35d4233ba23f09eba4fa74bccc3ef40a: 2023-02-13 00:56:27,219 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a., storeName=35d4233ba23f09eba4fa74bccc3ef40a/cf1, priority=9, startTime=1676249787177; duration=0sec 2023-02-13 00:56:27,219 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:27,219 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-top 2023-02-13 00:56:27,224 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-top 2023-02-13 00:56:27,224 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(310): Store=03c1550deb9567a2c004d092a0fcbbc1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:27,224 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,225 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2 2023-02-13 00:56:27,225 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2 2023-02-13 00:56:27,225 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 03c1550deb9567a2c004d092a0fcbbc1 columnFamilyName cf2 2023-02-13 00:56:27,236 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-top 2023-02-13 00:56:27,238 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-top 2023-02-13 00:56:27,241 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-top 2023-02-13 00:56:27,244 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-top 2023-02-13 00:56:27,249 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-top 2023-02-13 00:56:27,252 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-top 2023-02-13 00:56:27,254 DEBUG [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(539): loaded hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-top 2023-02-13 00:56:27,254 INFO [StoreOpener-03c1550deb9567a2c004d092a0fcbbc1-1] regionserver.HStore(310): Store=03c1550deb9567a2c004d092a0fcbbc1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 00:56:27,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 00:56:27,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 03c1550deb9567a2c004d092a0fcbbc1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59334339, jitterRate=-0.11584945023059845}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 00:56:27,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 03c1550deb9567a2c004d092a0fcbbc1: Running coprocessor pre-open hook at 1676249787180Writing region info on filesystem at 1676249787180Initializing all the Stores at 1676249787181 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249787181Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676249787181Cleaning up temporary data from old regions at 1676249787256 (+75 ms)Cleaning up detritus from prior splits at 1676249787257 (+1 ms)Running coprocessor post-open hooks at 1676249787258 (+1 ms)Region opened successfully at 1676249787258 2023-02-13 00:56:27,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1., pid=150, masterSystemTime=1676249787103 2023-02-13 00:56:27,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:27,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:27,260 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:27,260 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,260 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1912): 03c1550deb9567a2c004d092a0fcbbc1/cf1 is initiating minor compaction (all files) 2023-02-13 00:56:27,260 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 03c1550deb9567a2c004d092a0fcbbc1/cf1 in testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,261 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/.tmp, totalSize=34.9 K 2023-02-13 00:56:27,261 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249783782 2023-02-13 00:56:27,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,262 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249783950 2023-02-13 00:56:27,262 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=03c1550deb9567a2c004d092a0fcbbc1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:27,262 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787262"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676249787262"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676249787262"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676249787262"}]},"ts":"1676249787262"} 2023-02-13 00:56:27,262 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249784120 2023-02-13 00:56:27,263 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249784684 2023-02-13 00:56:27,263 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249784866 2023-02-13 00:56:27,264 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249785846 2023-02-13 00:56:27,264 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249786030 2023-02-13 00:56:27,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-13 00:56:27,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 03c1550deb9567a2c004d092a0fcbbc1, server=jenkins-hbase4.apache.org,43727,1676249678372 in 308 msec 2023-02-13 00:56:27,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-13 00:56:27,267 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, ASSIGN in 469 msec 2023-02-13 00:56:27,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=7a35765982dbe0f471877a25d912db99, daughterA=35d4233ba23f09eba4fa74bccc3ef40a, daughterB=03c1550deb9567a2c004d092a0fcbbc1 in 1.2300 sec 2023-02-13 00:56:27,272 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03c1550deb9567a2c004d092a0fcbbc1#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:27,281 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/.tmp/cf1/0943443396ed45e788b8d777d5197699 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/0943443396ed45e788b8d777d5197699 2023-02-13 00:56:27,285 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 03c1550deb9567a2c004d092a0fcbbc1/cf1 of 03c1550deb9567a2c004d092a0fcbbc1 into 0943443396ed45e788b8d777d5197699(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:27,285 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 03c1550deb9567a2c004d092a0fcbbc1: 2023-02-13 00:56:27,285 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1., storeName=03c1550deb9567a2c004d092a0fcbbc1/cf1, priority=9, startTime=1676249787259; duration=0sec 2023-02-13 00:56:27,285 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 00:56:27,285 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 00:56:27,285 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,286 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1912): 03c1550deb9567a2c004d092a0fcbbc1/cf2 is initiating minor compaction (all files) 2023-02-13 00:56:27,286 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 03c1550deb9567a2c004d092a0fcbbc1/cf2 in testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,286 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-top] into tmpdir=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/.tmp, totalSize=34.9 K 2023-02-13 00:56:27,286 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676249783782 2023-02-13 00:56:27,286 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676249783950 2023-02-13 00:56:27,287 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676249784120 2023-02-13 00:56:27,287 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676249784684 2023-02-13 00:56:27,287 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676249784866 2023-02-13 00:56:27,288 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676249785846 2023-02-13 00:56:27,288 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] compactions.Compactor(207): Compacting 73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676249786030 2023-02-13 00:56:27,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43727] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:56580 deadline: 1676249847288, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99. is not online on jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:27,297 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 03c1550deb9567a2c004d092a0fcbbc1#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 00:56:27,313 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/.tmp/cf2/58f88459b3774e5c9bb7736290933b09 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/58f88459b3774e5c9bb7736290933b09 2023-02-13 00:56:27,317 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 03c1550deb9567a2c004d092a0fcbbc1/cf2 of 03c1550deb9567a2c004d092a0fcbbc1 into 58f88459b3774e5c9bb7736290933b09(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:27,317 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 03c1550deb9567a2c004d092a0fcbbc1: 2023-02-13 00:56:27,318 INFO [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1., storeName=03c1550deb9567a2c004d092a0fcbbc1/cf2, priority=9, startTime=1676249787260; duration=0sec 2023-02-13 00:56:27,318 DEBUG [RS:0;jenkins-hbase4:43727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:27,397 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:56:27,404 INFO [Listener at localhost/43763] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 00:56:27,406 DEBUG [Listener at localhost/43763] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 00:56:27,422 WARN [Listener at localhost/43763] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 00:56:27,424 INFO [Listener at localhost/43763] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 00:56:27,424 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-13 00:56:27,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 00:56:27,428 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249787428"}]},"ts":"1676249787428"} 2023-02-13 00:56:27,429 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 00:56:27,431 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 00:56:27,431 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, UNASSIGN}] 2023-02-13 00:56:27,432 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, UNASSIGN 2023-02-13 00:56:27,432 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=03c1550deb9567a2c004d092a0fcbbc1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:27,433 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787432"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249787432"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249787432"}]},"ts":"1676249787432"} 2023-02-13 00:56:27,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 03c1550deb9567a2c004d092a0fcbbc1, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:27,435 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, UNASSIGN 2023-02-13 00:56:27,436 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=35d4233ba23f09eba4fa74bccc3ef40a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:27,436 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787436"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676249787436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676249787436"}]},"ts":"1676249787436"} 2023-02-13 00:56:27,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 35d4233ba23f09eba4fa74bccc3ef40a, server=jenkins-hbase4.apache.org,43727,1676249678372}] 2023-02-13 00:56:27,529 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 00:56:27,585 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 35d4233ba23f09eba4fa74bccc3ef40a, disabling compactions & flushes 2023-02-13 00:56:27,585 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,619 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/.tmp/cf2/1ed3eca656ec4e12b9d01c93c92fa384 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/1ed3eca656ec4e12b9d01c93c92fa384 2023-02-13 00:56:27,623 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 35d4233ba23f09eba4fa74bccc3ef40a/cf2 of 35d4233ba23f09eba4fa74bccc3ef40a into 1ed3eca656ec4e12b9d01c93c92fa384(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 00:56:27,623 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 35d4233ba23f09eba4fa74bccc3ef40a: 2023-02-13 00:56:27,623 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,623 INFO [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a., storeName=35d4233ba23f09eba4fa74bccc3ef40a/cf2, priority=9, startTime=1676249787178; duration=0sec 2023-02-13 00:56:27,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. after waiting 0 ms 2023-02-13 00:56:27,623 DEBUG [RS:0;jenkins-hbase4:43727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 00:56:27,623 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,624 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-bottom] to archive 2023-02-13 00:56:27,625 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:27,626 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,628 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,629 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,630 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,631 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,632 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,633 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,635 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-bottom, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-bottom] to archive 2023-02-13 00:56:27,636 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:27,637 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,638 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,639 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,640 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,641 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,642 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,642 DEBUG [StoreCloser-testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:56:27,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:27,646 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a. 2023-02-13 00:56:27,646 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 35d4233ba23f09eba4fa74bccc3ef40a: Waiting for close lock at 1676249787585Running coprocessor pre-close hooks at 1676249787585Disabling compacts and flushes for region at 1676249787585Disabling writes for close at 1676249787623 (+38 ms)Writing region close event to WAL at 1676249787644 (+21 ms)Running coprocessor post-close hooks at 1676249787646 (+2 ms)Closed at 1676249787646 2023-02-13 00:56:27,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 03c1550deb9567a2c004d092a0fcbbc1, disabling compactions & flushes 2023-02-13 00:56:27,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=35d4233ba23f09eba4fa74bccc3ef40a, regionState=CLOSED 2023-02-13 00:56:27,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. after waiting 0 ms 2023-02-13 00:56:27,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787648"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249787648"}]},"ts":"1676249787648"} 2023-02-13 00:56:27,649 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720-top] to archive 2023-02-13 00:56:27,650 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:27,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 00:56:27,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 35d4233ba23f09eba4fa74bccc3ef40a, server=jenkins-hbase4.apache.org,43727,1676249678372 in 213 msec 2023-02-13 00:56:27,652 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/1b4b15ec3e44480cb16ab29c86bbb765.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,652 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=35d4233ba23f09eba4fa74bccc3ef40a, UNASSIGN in 220 msec 2023-02-13 00:56:27,653 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,654 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/72eca8eda66445e1960e6694485350c6.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,655 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/3106e5b287444248bb029a6eec28ab9d.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,656 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/e45b337d824043d6b8564584ed3b432e.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,657 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/fc0a5bc1d34c435ba0f7a003b4de229d.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,658 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/f56e82f8eb2b43928c6aa41327414720.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,659 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a-top, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99->hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649-top] to archive 2023-02-13 00:56:27,660 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 00:56:27,661 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/d818351635d1415699fba25d955955b7.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,662 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/bec0a1192c714c01839b9306c3d0710c.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,663 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/2869547754ef431390468c7117c1e2c0.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,664 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/c15b3cd105484fd2a9fa197e8be194f4.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,665 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/38c5eac27f0142adb269dc2da3a35bba.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,666 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/e2816a2f0f57411e875c92d073ebec7a.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,667 DEBUG [StoreCloser-testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/73ba3e7ae97c442b8f48ae4af86f6649.7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 00:56:27,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:27,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1. 2023-02-13 00:56:27,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 03c1550deb9567a2c004d092a0fcbbc1: Waiting for close lock at 1676249787648Running coprocessor pre-close hooks at 1676249787648Disabling compacts and flushes for region at 1676249787648Disabling writes for close at 1676249787648Writing region close event to WAL at 1676249787668 (+20 ms)Running coprocessor post-close hooks at 1676249787671 (+3 ms)Closed at 1676249787671 2023-02-13 00:56:27,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,673 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=03c1550deb9567a2c004d092a0fcbbc1, regionState=CLOSED 2023-02-13 00:56:27,673 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676249787673"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676249787673"}]},"ts":"1676249787673"} 2023-02-13 00:56:27,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 00:56:27,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 03c1550deb9567a2c004d092a0fcbbc1, server=jenkins-hbase4.apache.org,43727,1676249678372 in 241 msec 2023-02-13 00:56:27,676 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 00:56:27,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 00:56:27,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=03c1550deb9567a2c004d092a0fcbbc1, UNASSIGN in 244 msec 2023-02-13 00:56:27,678 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676249787678"}]},"ts":"1676249787678"} 2023-02-13 00:56:27,679 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 00:56:27,682 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 00:56:27,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 263 msec 2023-02-13 00:56:27,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 00:56:27,749 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 00:56:27,750 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-13 00:56:27,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,758 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 00:56:27,763 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,763 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,763 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,766 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/recovered.edits] 2023-02-13 00:56:27,766 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/recovered.edits] 2023-02-13 00:56:27,767 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2, FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/recovered.edits] 2023-02-13 00:56:27,769 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/0943443396ed45e788b8d777d5197699 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf1/0943443396ed45e788b8d777d5197699 2023-02-13 00:56:27,770 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/4a79047474964908a2020aaedc0445fe to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf1/4a79047474964908a2020aaedc0445fe 2023-02-13 00:56:27,770 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/1b4b15ec3e44480cb16ab29c86bbb765 2023-02-13 00:56:27,772 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/3106e5b287444248bb029a6eec28ab9d 2023-02-13 00:56:27,773 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/58f88459b3774e5c9bb7736290933b09 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/cf2/58f88459b3774e5c9bb7736290933b09 2023-02-13 00:56:27,773 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/1ed3eca656ec4e12b9d01c93c92fa384 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/cf2/1ed3eca656ec4e12b9d01c93c92fa384 2023-02-13 00:56:27,774 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/72eca8eda66445e1960e6694485350c6 2023-02-13 00:56:27,775 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/e45b337d824043d6b8564584ed3b432e 2023-02-13 00:56:27,776 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a/recovered.edits/90.seqid 2023-02-13 00:56:27,776 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/recovered.edits/90.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1/recovered.edits/90.seqid 2023-02-13 00:56:27,777 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/35d4233ba23f09eba4fa74bccc3ef40a 2023-02-13 00:56:27,777 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/03c1550deb9567a2c004d092a0fcbbc1 2023-02-13 00:56:27,777 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/f56e82f8eb2b43928c6aa41327414720 2023-02-13 00:56:27,778 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fc0a5bc1d34c435ba0f7a003b4de229d 2023-02-13 00:56:27,779 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf1/fce0b2f5c9ea43e7a6a58412ad0f5e20 2023-02-13 00:56:27,782 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/2869547754ef431390468c7117c1e2c0 2023-02-13 00:56:27,783 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/38c5eac27f0142adb269dc2da3a35bba 2023-02-13 00:56:27,784 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/73ba3e7ae97c442b8f48ae4af86f6649 2023-02-13 00:56:27,785 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/bec0a1192c714c01839b9306c3d0710c 2023-02-13 00:56:27,786 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/c15b3cd105484fd2a9fa197e8be194f4 2023-02-13 00:56:27,788 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7 to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/d818351635d1415699fba25d955955b7 2023-02-13 00:56:27,789 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/cf2/e2816a2f0f57411e875c92d073ebec7a 2023-02-13 00:56:27,797 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/recovered.edits/85.seqid to hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/archive/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99/recovered.edits/85.seqid 2023-02-13 00:56:27,797 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/.tmp/data/default/testSplitTableRegion/7a35765982dbe0f471877a25d912db99 2023-02-13 00:56:27,797 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 00:56:27,799 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,801 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 00:56:27,803 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 00:56:27,804 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,804 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 00:56:27,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249787804"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:27,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249787804"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:27,804 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676249787804"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:27,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 00:56:27,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7a35765982dbe0f471877a25d912db99, NAME => 'testSplitTableRegion,,1676249783278.7a35765982dbe0f471877a25d912db99.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 35d4233ba23f09eba4fa74bccc3ef40a, NAME => 'testSplitTableRegion,,1676249786037.35d4233ba23f09eba4fa74bccc3ef40a.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 03c1550deb9567a2c004d092a0fcbbc1, NAME => 'testSplitTableRegion,41,1676249786037.03c1550deb9567a2c004d092a0fcbbc1.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 00:56:27,806 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 00:56:27,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676249787806"}]},"ts":"9223372036854775807"} 2023-02-13 00:56:27,808 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 00:56:27,809 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 00:56:27,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 60 msec 2023-02-13 00:56:27,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43619] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 00:56:27,860 INFO [Listener at localhost/43763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 00:56:27,876 INFO [Listener at localhost/43763] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377 (was 372) - Thread LEAK? -, OpenFileDescriptor=834 (was 850), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=143 (was 103) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=3190 (was 3217) 2023-02-13 00:56:27,876 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 00:56:27,880 INFO [Listener at localhost/43763] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 00:56:27,880 DEBUG [Listener at localhost/43763] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x328d140f to 127.0.0.1:63852 2023-02-13 00:56:27,880 DEBUG [Listener at localhost/43763] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:27,881 DEBUG [Listener at localhost/43763] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 00:56:27,881 DEBUG [Listener at localhost/43763] util.JVMClusterUtil(257): Found active master hash=115348766, stopped=false 2023-02-13 00:56:27,881 INFO [Listener at localhost/43763] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:56:27,883 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:27,883 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:27,883 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:27,883 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:56:27,883 INFO [Listener at localhost/43763] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 00:56:27,883 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 00:56:27,883 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:27,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:27,884 DEBUG [Listener at localhost/43763] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x48e92653 to 127.0.0.1:63852 2023-02-13 00:56:27,884 DEBUG [Listener at localhost/43763] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:27,884 INFO [Listener at localhost/43763] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43727,1676249678372' ***** 2023-02-13 00:56:27,884 INFO [Listener at localhost/43763] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 00:56:27,884 INFO [Listener at localhost/43763] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,34557,1676249678414' ***** 2023-02-13 00:56:27,884 INFO [Listener at localhost/43763] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 00:56:27,884 INFO [Listener at localhost/43763] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41539,1676249678442' ***** 2023-02-13 00:56:27,885 INFO [RS:0;jenkins-hbase4:43727] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 00:56:27,885 INFO [RS:1;jenkins-hbase4:34557] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 00:56:27,885 INFO [Listener at localhost/43763] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 00:56:27,885 INFO [RS:0;jenkins-hbase4:43727] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 00:56:27,885 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 00:56:27,885 INFO [RS:2;jenkins-hbase4:41539] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 00:56:28,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:27,885 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 00:56:27,885 INFO [RS:1;jenkins-hbase4:34557] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 00:56:28,052 INFO [RS:1;jenkins-hbase4:34557] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 00:56:28,052 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:28,052 INFO [RS:2;jenkins-hbase4:41539] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 00:56:28,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 00:56:28,052 INFO [RS:2;jenkins-hbase4:41539] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 00:56:28,051 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,051 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,051 INFO [RS:0;jenkins-hbase4:43727] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 00:56:28,053 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(3304): Received CLOSE for 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:56:28,051 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,051 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 00:56:28,053 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:28,053 DEBUG [RS:1;jenkins-hbase4:34557] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64cdf83b to 127.0.0.1:63852 2023-02-13 00:56:28,053 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:28,053 DEBUG [RS:0;jenkins-hbase4:43727] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32babb26 to 127.0.0.1:63852 2023-02-13 00:56:28,053 DEBUG [RS:2;jenkins-hbase4:41539] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5ceda779 to 127.0.0.1:63852 2023-02-13 00:56:28,053 DEBUG [RS:1;jenkins-hbase4:34557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,053 DEBUG [RS:2;jenkins-hbase4:41539] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,053 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,34557,1676249678414; all regions closed. 2023-02-13 00:56:28,053 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 00:56:28,053 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2a8283ab9ffc3d4f3875786a1eb0696d, disabling compactions & flushes 2023-02-13 00:56:28,053 DEBUG [RS:0;jenkins-hbase4:43727] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:56:28,054 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1479): Online Regions={2a8283ab9ffc3d4f3875786a1eb0696d=hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d.} 2023-02-13 00:56:28,054 INFO [RS:0;jenkins-hbase4:43727] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 00:56:28,054 INFO [RS:0;jenkins-hbase4:43727] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 00:56:28,054 INFO [RS:0;jenkins-hbase4:43727] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:56:28,054 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. after waiting 0 ms 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:56:28,054 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2a8283ab9ffc3d4f3875786a1eb0696d 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 00:56:28,054 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 00:56:28,054 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 00:56:28,054 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 00:56:28,054 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 00:56:28,055 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 00:56:28,055 DEBUG [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 00:56:28,055 DEBUG [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1505): Waiting on 2a8283ab9ffc3d4f3875786a1eb0696d 2023-02-13 00:56:28,055 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-13 00:56:28,064 DEBUG [RS:1;jenkins-hbase4:34557] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs 2023-02-13 00:56:28,064 INFO [RS:1;jenkins-hbase4:34557] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34557%2C1676249678414:(num 1676249679844) 2023-02-13 00:56:28,064 DEBUG [RS:1;jenkins-hbase4:34557] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,064 INFO [RS:1;jenkins-hbase4:34557] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,064 INFO [RS:1;jenkins-hbase4:34557] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-13 00:56:28,065 INFO [RS:1;jenkins-hbase4:34557] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 00:56:28,065 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:28,065 INFO [RS:1;jenkins-hbase4:34557] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 00:56:28,065 INFO [RS:1;jenkins-hbase4:34557] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 00:56:28,071 INFO [RS:1;jenkins-hbase4:34557] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34557 2023-02-13 00:56:28,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/.tmp/info/93d7f05be1de4f0fac1d8250312c76fe 2023-02-13 00:56:28,086 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:28,086 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:28,086 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34557,1676249678414 2023-02-13 00:56:28,086 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,086 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,088 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@20f31fee rejected from java.util.concurrent.ThreadPoolExecutor@24cc9f28[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,088 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,088 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6851bb80 rejected from java.util.concurrent.ThreadPoolExecutor@24cc9f28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,088 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/info/df695676c47c4514a41a8c46b9fb3b1b 2023-02-13 00:56:28,089 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,090 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34557,1676249678414] 2023-02-13 00:56:28,090 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34557,1676249678414; numProcessing=1 2023-02-13 00:56:28,091 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34557,1676249678414 already deleted, retry=false 2023-02-13 00:56:28,091 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34557,1676249678414 expired; onlineServers=2 2023-02-13 00:56:28,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/.tmp/info/93d7f05be1de4f0fac1d8250312c76fe as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/info/93d7f05be1de4f0fac1d8250312c76fe 2023-02-13 00:56:28,092 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df695676c47c4514a41a8c46b9fb3b1b 2023-02-13 00:56:28,094 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/info/93d7f05be1de4f0fac1d8250312c76fe, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 00:56:28,096 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2a8283ab9ffc3d4f3875786a1eb0696d in 42ms, sequenceid=6, compaction requested=false 2023-02-13 00:56:28,104 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/namespace/2a8283ab9ffc3d4f3875786a1eb0696d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 00:56:28,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:28,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:56:28,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2a8283ab9ffc3d4f3875786a1eb0696d: Waiting for close lock at 1676249788053Running coprocessor pre-close hooks at 1676249788053Disabling compacts and flushes for region at 1676249788053Disabling writes for close at 1676249788054 (+1 ms)Obtaining lock to block concurrent updates at 1676249788054Preparing flush snapshotting stores in 2a8283ab9ffc3d4f3875786a1eb0696d at 1676249788054Finished memstore snapshotting hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676249788054Flushing stores of hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. at 1676249788055 (+1 ms)Flushing 2a8283ab9ffc3d4f3875786a1eb0696d/info: creating writer at 1676249788055Flushing 2a8283ab9ffc3d4f3875786a1eb0696d/info: appending metadata at 1676249788062 (+7 ms)Flushing 2a8283ab9ffc3d4f3875786a1eb0696d/info: closing flushed file at 1676249788062Flushing 2a8283ab9ffc3d4f3875786a1eb0696d/info: reopening flushed file at 1676249788092 (+30 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 2a8283ab9ffc3d4f3875786a1eb0696d in 42ms, sequenceid=6, compaction requested=false at 1676249788096 (+4 ms)Writing region close event to WAL at 1676249788098 (+2 ms)Running coprocessor post-close hooks at 1676249788105 (+7 ms)Closed at 1676249788106 (+1 ms) 2023-02-13 00:56:28,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676249680322.2a8283ab9ffc3d4f3875786a1eb0696d. 2023-02-13 00:56:28,107 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/rep_barrier/0b81edde758f4d9885d557ad6f90cdf2 2023-02-13 00:56:28,111 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b81edde758f4d9885d557ad6f90cdf2 2023-02-13 00:56:28,122 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/table/8a9f974cf5494e61a0f25e53278e4923 2023-02-13 00:56:28,126 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8a9f974cf5494e61a0f25e53278e4923 2023-02-13 00:56:28,126 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/info/df695676c47c4514a41a8c46b9fb3b1b as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/info/df695676c47c4514a41a8c46b9fb3b1b 2023-02-13 00:56:28,130 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for df695676c47c4514a41a8c46b9fb3b1b 2023-02-13 00:56:28,130 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/info/df695676c47c4514a41a8c46b9fb3b1b, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 00:56:28,130 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/rep_barrier/0b81edde758f4d9885d557ad6f90cdf2 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/rep_barrier/0b81edde758f4d9885d557ad6f90cdf2 2023-02-13 00:56:28,133 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0b81edde758f4d9885d557ad6f90cdf2 2023-02-13 00:56:28,133 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/rep_barrier/0b81edde758f4d9885d557ad6f90cdf2, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 00:56:28,134 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/.tmp/table/8a9f974cf5494e61a0f25e53278e4923 as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/table/8a9f974cf5494e61a0f25e53278e4923 2023-02-13 00:56:28,137 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8a9f974cf5494e61a0f25e53278e4923 2023-02-13 00:56:28,138 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/table/8a9f974cf5494e61a0f25e53278e4923, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 00:56:28,138 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 83ms, sequenceid=210, compaction requested=false 2023-02-13 00:56:28,143 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 00:56:28,143 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 00:56:28,143 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 00:56:28,144 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 00:56:28,145 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676249788054Running coprocessor pre-close hooks at 1676249788054Disabling compacts and flushes for region at 1676249788054Disabling writes for close at 1676249788055 (+1 ms)Obtaining lock to block concurrent updates at 1676249788055Preparing flush snapshotting stores in 1588230740 at 1676249788055Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1676249788055Flushing stores of hbase:meta,,1.1588230740 at 1676249788055Flushing 1588230740/info: creating writer at 1676249788055Flushing 1588230740/info: appending metadata at 1676249788065 (+10 ms)Flushing 1588230740/info: closing flushed file at 1676249788065Flushing 1588230740/rep_barrier: creating writer at 1676249788093 (+28 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676249788095 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676249788095Flushing 1588230740/table: creating writer at 1676249788111 (+16 ms)Flushing 1588230740/table: appending metadata at 1676249788113 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676249788113Flushing 1588230740/info: reopening flushed file at 1676249788127 (+14 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676249788131 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676249788134 (+3 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 83ms, sequenceid=210, compaction requested=false at 1676249788138 (+4 ms)Writing region close event to WAL at 1676249788140 (+2 ms)Running coprocessor post-close hooks at 1676249788143 (+3 ms)Closed at 1676249788144 (+1 ms) 2023-02-13 00:56:28,145 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 00:56:28,190 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,190 INFO [RS:1;jenkins-hbase4:34557] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,34557,1676249678414; zookeeper connection closed. 2023-02-13 00:56:28,190 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@109025e0 rejected from java.util.concurrent.ThreadPoolExecutor@24cc9f28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,190 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:34557-0x1004b2c240b0002, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,190 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6489cc9d rejected from java.util.concurrent.ThreadPoolExecutor@24cc9f28[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,190 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@24f9761b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@24f9761b 2023-02-13 00:56:28,255 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43727,1676249678372; all regions closed. 2023-02-13 00:56:28,255 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41539,1676249678442; all regions closed. 2023-02-13 00:56:28,263 DEBUG [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs 2023-02-13 00:56:28,263 INFO [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43727%2C1676249678372.meta:.meta(num 1676249680053) 2023-02-13 00:56:28,263 DEBUG [RS:2;jenkins-hbase4:41539] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs 2023-02-13 00:56:28,263 INFO [RS:2;jenkins-hbase4:41539] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41539%2C1676249678442:(num 1676249679845) 2023-02-13 00:56:28,263 DEBUG [RS:2;jenkins-hbase4:41539] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,263 INFO [RS:2;jenkins-hbase4:41539] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,268 INFO [RS:2;jenkins-hbase4:41539] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 00:56:28,269 INFO [RS:2;jenkins-hbase4:41539] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 00:56:28,269 INFO [RS:2;jenkins-hbase4:41539] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 00:56:28,269 INFO [RS:2;jenkins-hbase4:41539] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 00:56:28,270 INFO [RS:2;jenkins-hbase4:41539] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41539 2023-02-13 00:56:28,270 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:28,272 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,272 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41539,1676249678442] 2023-02-13 00:56:28,273 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41539,1676249678442; numProcessing=2 2023-02-13 00:56:28,275 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:28,276 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41539,1676249678442 2023-02-13 00:56:28,276 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41539,1676249678442 already deleted, retry=false 2023-02-13 00:56:28,277 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41539,1676249678442 expired; onlineServers=1 2023-02-13 00:56:28,283 DEBUG [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/oldWALs 2023-02-13 00:56:28,283 INFO [RS:0;jenkins-hbase4:43727] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C43727%2C1676249678372:(num 1676249679846) 2023-02-13 00:56:28,283 DEBUG [RS:0;jenkins-hbase4:43727] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,283 INFO [RS:0;jenkins-hbase4:43727] regionserver.LeaseManager(133): Closed leases 2023-02-13 00:56:28,283 INFO [RS:0;jenkins-hbase4:43727] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 00:56:28,283 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:28,284 INFO [RS:0;jenkins-hbase4:43727] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43727 2023-02-13 00:56:28,285 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,43727,1676249678372 2023-02-13 00:56:28,285 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 00:56:28,285 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5998bbb4 rejected from java.util.concurrent.ThreadPoolExecutor@5da7245f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,286 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,43727,1676249678372] 2023-02-13 00:56:28,286 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,43727,1676249678372; numProcessing=3 2023-02-13 00:56:28,289 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,43727,1676249678372 already deleted, retry=false 2023-02-13 00:56:28,289 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,43727,1676249678372 expired; onlineServers=0 2023-02-13 00:56:28,289 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43619,1676249677220' ***** 2023-02-13 00:56:28,289 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 00:56:28,290 DEBUG [M:0;jenkins-hbase4:43619] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47dd1f25, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-13 00:56:28,290 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43619,1676249677220 2023-02-13 00:56:28,290 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43619,1676249677220; all regions closed. 2023-02-13 00:56:28,290 DEBUG [M:0;jenkins-hbase4:43619] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 00:56:28,290 DEBUG [M:0;jenkins-hbase4:43619] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 00:56:28,290 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 00:56:28,290 DEBUG [M:0;jenkins-hbase4:43619] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 00:56:28,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249679510] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676249679510,5,FailOnTimeoutGroup] 2023-02-13 00:56:28,290 INFO [M:0;jenkins-hbase4:43619] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 00:56:28,290 INFO [M:0;jenkins-hbase4:43619] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 00:56:28,290 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249679513] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676249679513,5,FailOnTimeoutGroup] 2023-02-13 00:56:28,290 INFO [M:0;jenkins-hbase4:43619] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-13 00:56:28,291 DEBUG [M:0;jenkins-hbase4:43619] master.HMaster(1502): Stopping service threads 2023-02-13 00:56:28,291 INFO [M:0;jenkins-hbase4:43619] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 00:56:28,291 ERROR [M:0;jenkins-hbase4:43619] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 00:56:28,291 INFO [M:0;jenkins-hbase4:43619] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 00:56:28,291 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 00:56:28,292 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 00:56:28,292 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 00:56:28,292 DEBUG [M:0;jenkins-hbase4:43619] zookeeper.ZKUtil(398): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 00:56:28,292 WARN [M:0;jenkins-hbase4:43619] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 00:56:28,292 INFO [M:0;jenkins-hbase4:43619] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 00:56:28,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 00:56:28,293 INFO [M:0;jenkins-hbase4:43619] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 00:56:28,293 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 00:56:28,293 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:28,293 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:28,293 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 00:56:28,293 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:28,293 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.98 KB heapSize=718.99 KB 2023-02-13 00:56:28,318 INFO [M:0;jenkins-hbase4:43619] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.98 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ebfa7294fa4e4c3281b267752fe9051a 2023-02-13 00:56:28,322 INFO [M:0;jenkins-hbase4:43619] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebfa7294fa4e4c3281b267752fe9051a 2023-02-13 00:56:28,323 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ebfa7294fa4e4c3281b267752fe9051a as hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ebfa7294fa4e4c3281b267752fe9051a 2023-02-13 00:56:28,326 INFO [M:0;jenkins-hbase4:43619] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebfa7294fa4e4c3281b267752fe9051a 2023-02-13 00:56:28,327 INFO [M:0;jenkins-hbase4:43619] regionserver.HStore(1080): Added hdfs://localhost:41999/user/jenkins/test-data/b247760e-c289-2594-e292-1aeb86b1e3ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ebfa7294fa4e4c3281b267752fe9051a, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 00:56:28,327 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegion(2947): Finished flush of dataSize ~607.98 KB/622576, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=1282, compaction requested=false 2023-02-13 00:56:28,328 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 00:56:28,328 DEBUG [M:0;jenkins-hbase4:43619] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676249788293Disabling compacts and flushes for region at 1676249788293Disabling writes for close at 1676249788293Obtaining lock to block concurrent updates at 1676249788293Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676249788293Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622576, getHeapSize=736232, getOffHeapSize=0, getCellsCount=1499 at 1676249788293Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676249788299 (+6 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676249788299Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676249788303 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676249788303Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676249788323 (+20 ms)Finished flush of dataSize ~607.98 KB/622576, heapSize ~718.98 KB/736232, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 34ms, sequenceid=1282, compaction requested=false at 1676249788328 (+5 ms)Writing region close event to WAL at 1676249788328Closed at 1676249788328 2023-02-13 00:56:28,346 INFO [M:0;jenkins-hbase4:43619] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 00:56:28,346 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 00:56:28,347 INFO [M:0;jenkins-hbase4:43619] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43619 2023-02-13 00:56:28,348 DEBUG [M:0;jenkins-hbase4:43619] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43619,1676249677220 already deleted, retry=false 2023-02-13 00:56:28,380 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,380 INFO [RS:2;jenkins-hbase4:41539] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41539,1676249678442; zookeeper connection closed. 2023-02-13 00:56:28,380 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@772b8fc0 rejected from java.util.concurrent.ThreadPoolExecutor@731405fd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,380 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:41539-0x1004b2c240b0003, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,380 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1d687a1c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1d687a1c 2023-02-13 00:56:28,380 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2c141db2 rejected from java.util.concurrent.ThreadPoolExecutor@731405fd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,480 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,480 INFO [M:0;jenkins-hbase4:43619] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43619,1676249677220; zookeeper connection closed. 2023-02-13 00:56:28,480 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1e627fc7 rejected from java.util.concurrent.ThreadPoolExecutor@593ac078[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,480 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): master:43619-0x1004b2c240b0000, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,481 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@40994993 rejected from java.util.concurrent.ThreadPoolExecutor@593ac078[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,580 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,580 INFO [RS:0;jenkins-hbase4:43727] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43727,1676249678372; zookeeper connection closed. 2023-02-13 00:56:28,581 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@352eb383 rejected from java.util.concurrent.ThreadPoolExecutor@5da7245f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,581 DEBUG [Listener at localhost/43763-EventThread] zookeeper.ZKWatcher(600): regionserver:43727-0x1004b2c240b0001, quorum=127.0.0.1:63852, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 00:56:28,581 ERROR [Listener at localhost/43763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4d16b724 rejected from java.util.concurrent.ThreadPoolExecutor@5da7245f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 00:56:28,581 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2ae5bf8] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2ae5bf8 2023-02-13 00:56:28,581 INFO [Listener at localhost/43763] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 00:56:28,582 WARN [Listener at localhost/43763] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 00:56:28,586 INFO [Listener at localhost/43763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:28,691 WARN [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 00:56:28,691 WARN [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid f25dc2b5-f59c-472e-873a-01e1c6fd94f9) service to localhost/127.0.0.1:41999 2023-02-13 00:56:28,694 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data5/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,695 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data6/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,697 WARN [Listener at localhost/43763] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 00:56:28,703 INFO [Listener at localhost/43763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:28,705 WARN [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid 4e877f44-49cc-41c6-9a12-77167167aa83) service to localhost/127.0.0.1:41999 2023-02-13 00:56:28,706 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data3/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,707 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data4/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,709 WARN [Listener at localhost/43763] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 00:56:28,722 INFO [Listener at localhost/43763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:28,727 WARN [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 00:56:28,728 WARN [BP-263835449-172.31.14.131-1676249674116 heartbeating to localhost/127.0.0.1:41999] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-263835449-172.31.14.131-1676249674116 (Datanode Uuid 5c1a259a-c33c-46e5-8622-2360d3fa2b07) service to localhost/127.0.0.1:41999 2023-02-13 00:56:28,735 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data1/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,736 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/7f6f1875-f4eb-3929-a482-e75d5b5bbab5/cluster_b94a0796-5ab7-e038-552c-bc2a7822da2c/dfs/data/data2/current/BP-263835449-172.31.14.131-1676249674116] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 00:56:28,760 INFO [Listener at localhost/43763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 00:56:28,952 INFO [Listener at localhost/43763] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 00:56:29,179 INFO [Listener at localhost/43763] hbase.HBaseTestingUtility(1293): Minicluster is down